Intraocular lens simulator using computational holographic display for cataract patients.

PURPOSE: To develop and validate a holography based vision simulator for the demonstration of expected postoperative vision corresponding to monofocal and multifocal intraocular lenses (IOL) to cataract patients before surgery. METHODS: An artificial eye model is used to measure the optical performance of different IOL types. The resultant aberrations and degradations are then modeled using phase holograms and shown to subjects on a holographic display. We measure the contrast and resolution loss, halos around the light sources, and point spread function (PSF) corresponding to three different IOLs. We tested the holography based vision simulator on 13 healthy subjects and 6 cataract patients. RESULTS: Monofocal, bifocal, and trifocal IOLs exhibited a contrast decrease of 5%, 42%, and 45% and a resolution limit of 4.49, 4.00, and 4.00 lp/mm (using 0.05 MTF criteria), respectively. Monofocal IOLs have the best resolution and contrast at the optimal focus distance, and multifocal lenses offer extended depth-of-field but exhibit prominent halos and reduced contrast/resolution. CONCLUSION: We confirmed that the visual functions of IOLs could be successfully modeled using phase holograms and simulated using a holographic display without using a physical IOL. Patients can experience the effects of different IOL options prior to surgery, which helps with IOL selection, expectation management, and patient satisfaction.

Three-Dimensional Holographic-Guided Robotic Lung Segmentectomy for Deep Pulmonary Nodules: Technique and Initial Results.

Background: Diagnosis and treatment of small and isolated lung nodules remain challenging issues. Purpose: The aim of this article is to report the technique of real-time navigation using holographic reconstruction technology combined with a robot assisted thoracic surgery (RATS) platform for lung resection in patients with small deep nodules.Research Design: The pre-surgery 3D planning was based on the chest CT scan. The reconstruction was uploaded to a head-mounted display for real-time navigation during mini invasive robot assisted surgery performed with an open console platform. We evaluated this technique with the success rate of diagnosis, the operative time and the post-operative course.Study Sample: This technique was performed in 6 patients (4 female, mean age 65 years) to date.Results: The precision of the head-mounted display based localization system was effective in all cases without the need of open conversion. The mean diameter of the nodules was 8 mm (6-9). The diagnosis was a lung cancer (n = 5) and tuberculoma (n = 1). The mean operative time was 125 min (100-145). The mean hospital stay was 2.5 days (1-3).Conclusions: In conclusion, the intraoperative navigation using the 3D holographic assistance was an helpful tool for mini invasive RATS lung segmentectomy without the need of preoperative localization.

Mixed Reality Assisted Navigation Guided Microsurgical Removal of Cranial Lesions.

AIM: To demonstrate the possible use of mixed reality (MR) technology in neurosurgery for multiple purposes, including preoperative planning, training, and three-dimensional (3D) navigation. MATERIAL AND METHODS: Using magnetic resonance imaging (MRI) and computed tomography (CT), 3D holographic images of three patients were created and inspected using a remote control. Preoperative planning was performed in a conference room using holographic images. Intraoperatively, the 3D images were matched and the adjacent structures were examined. RESULTS: The MR System (MRS) was a useful tool for preoperative planning and intraoperative navigation during the cranial intervention. It reduces operative time, decreases complication rates, increases surgical success, and enhances surgical outcomes. Eventually, MRS may be more economical. CONCLUSION: The MRS can be used for intraoperative navigation by displaying a 3D hologram at the surgeon's fingertips and for preoperative 3D examination of the lesions and its surrounding structures. The MRS enhances surgical efficacy, reduces healthcare costs, and has a shorter learning curve than the conventional methods. It also enables customized patient-specific surgery.

A holographic telementoring system depicting surgical instrument movements for real-time guidance in open surgeries.

BACKGROUND AND OBJECTIVE: During open surgeries, telementoring serves as a valuable tool for transferring surgical knowledge from a specialist surgeon (mentor) to an operating surgeon (mentee). Depicting the intended movements of the surgical instruments over the operative field improves the understanding of the required tool-tissue interaction. The objective of this work is to develop a telementoring system tailored for open surgeries, enabling the mentor to remotely demonstrate the necessary motions of surgical instruments to the mentee. METHODS: A remote telementoring system for open surgery was implemented. The system generates visual cues in the form of virtual surgical instrument motion augmented onto the live view of the operative field. These cues can be rendered on both conventional screens in the operating room and as dynamic holograms on a head mounted display device worn by the mentee. The technical performance of the system was evaluated, where the operating room and remote location were geographically separated and connected via the Internet. Additionally, user studies were conducted to assess the effectiveness of the system as a mentoring tool. RESULTS: The system took 307 ± 12 ms to transmit an operative field view of 1920 × 1080 resolution, along with depth information spanning 36 cm, from the operating room to the remote location. Conversely, it took 145 ± 14 ms to receive the motion of virtual surgical instruments from the remote location back to the operating room. Furthermore, the user studies demonstrated: (a) mentor's capability to annotate the operative field with an accuracy of 3.92 ± 2.1 mm, (b) mentee's ability to comprehend and replicate the motion of surgical instruments in real-time with an average deviation of 12.8 ± 3 mm, (c) efficacy of the rendered dynamic holograms in conveying information intended for surgical instrument motion. CONCLUSIONS: The study demonstrates the feasibility of transmitting information over the Internet from the mentor to the mentee in the form of virtual surgical instruments' motion and projecting it as holograms onto the live view of the operative field. This holds potential to enhance real-time collaborative capabilities between the mentor and the mentee during an open surgery.

Off-axis digital lensless holographic microscopy based on spatially multiplexed interferometry.

Significance: Digital holographic microscopy (DHM) is a label-free microscopy technique that provides time-resolved quantitative phase imaging (QPI) by measuring the optical path delay of light induced by transparent biological samples. DHM has been utilized for various biomedical applications, such as cancer research and sperm cell assessment, as well as for in vitro drug or toxicity testing. Its lensless version, digital lensless holographic microscopy (DLHM), is an emerging technology that offers size-reduced, lightweight, and cost-effective imaging systems. These features make DLHM applicable, for example, in limited resource laboratories, remote areas, and point-of-care applications. Aim: In addition to the abovementioned advantages, in-line arrangements for DLHM also include the limitation of the twin-image presence, which can restrict accurate QPI. We therefore propose a compact lensless common-path interferometric off-axis approach that is capable of quantitative imaging of fast-moving biological specimens, such as living cells in flow. Approach: We suggest lensless spatially multiplexed interferometric microscopy (LESSMIM) as a lens-free variant of the previously reported spatially multiplexed interferometric microscopy (SMIM) concept. LESSMIM comprises a common-path interferometric architecture that is based on a single diffraction grating to achieve digital off-axis holography. From a series of single-shot off-axis holograms, twin-image free and time-resolved QPI is achieved by commonly used methods for Fourier filtering-based reconstruction, aberration compensation, and numerical propagation. Results: Initially, the LESSMIM concept is experimentally demonstrated by results from a resolution test chart and investigations on temporal stability. Then, the accuracy of QPI and capabilities for imaging of living adherent cell cultures is characterized. Finally, utilizing a microfluidic channel, the cytometry of suspended cells in flow is evaluated. Conclusions: LESSMIM overcomes several limitations of in-line DLHM and provides fast time-resolved QPI in a compact optical arrangement. In summary, LESSMIM represents a promising technique with potential biomedical applications for fast imaging such as in imaging flow cytometry or sperm cell analysis.

Implant placement using mixed reality-based dynamic navigation: A proof of concept.

OBJECTIVES: To present the first clinical application of a novel mixed reality-based dynamic navigation (MR-DN) system in the rehabilitation of a single tooth gap. METHODS: The protocol consisted of the following: (1) three-dimensional patient data acquisition using intraoral scanning (IOS) and cone-beam computed tomography (CBCT), (2) implant planning using guided surgery software, (3) holography-guided implant placement using the novel MR-DN system (ANNA®, MARS Dental, Haifa, Israel) and (4) placement accuracy verification. RESULTS: The novel MR-DN system was safe and time-efficient, as the surgery took 30 min from anaesthesia to suturing. The accuracy of implant placement was high with minimal deviations recorded in the three planes of space compared to the presurgical planning: the error at the entry point planar distance (XY) was 0.381 mm, and the entry point planar distance (Z) was 0.173 mm, for a 3D entry point distance (En) of 0.417 mm. A 3D apex deviation (An) of 0.193 mm was registered, with an angle difference of 1.852° Conclusions: This proof-of-concept study demonstrated the clinical feasibility of MR-DN for guided implant placement in single tooth gaps. Further clinical studies on a large sample of patients are needed to confirm these positive preliminary results. Statement of clinical relevance: The use of MR-DN can change the perspectives of guided dental implant surgery as a possible alternative to the classic static and dynamic guided surgical techniques for the rehabilitation of single tooth gaps.

Acoustic-holography-patterned primary hepatocytes possess liver functions.

Acoustic holography (AH), a promising approach for cell patterning, emerges as a powerful tool for constructing novel invitro 3D models that mimic organs and cancers features. However, understanding changes in cell function post-AH remains limited. Furthermore, replicating complex physiological and pathological processes solely with cell lines proves challenging. Here, we employed acoustical holographic lattice to assemble primary hepatocytes directly isolated from mice into a cell cluster matrix to construct a liver-shaped tissue sample. For the first time, we evaluated the liver functions of AH-patterned primary hepatocytes. The patterned model exhibited large numbers of self-assembled spheroids and superior multifarious core hepatocyte functions compared to cells in 2D and traditional 3D culture models. AH offers a robust protocol for long-term in vitro culture of primary cells, underscoring its potential for future applications in disease pathogenesis research, drug testing, and organ replacement therapy.

Use of a platform with lens-free shadow imaging technology to monitor natural killer cell activity.

Natural killer (NK) cells are a crucial component of the innate immune system. This study introduces Cellytics NK, a novel platform for rapid and precise measurement of NK cell activity. This platform combines an NK-specific activation stimulator cocktail (ASC) and lens-free shadow imaging technology (LSIT), using optoelectronic components. LSIT captures digital hologram images of resting and ASC-activated NK cells, while an algorithm evaluates cell size and cytoplasmic complexity using shadow parameters. The combined shadow parameter derived from the peak-to-peak distance and width standard deviation rapidly distinguishes active NK cells from inactive NK cells at the single-cell level within 30 s. Here, the feasibility of the system was demonstrated by assessing NK cells from healthy donors and immunocompromised cancer patients, demonstrating a significant difference in the innate immunity index (I3). Cancer patients showed a lower I3 value (161%) than healthy donors (326%). I3 was strongly correlated with NK cell activity measured using various markers such as interferon-gamma, tumor necrosis factor-alpha, perforin, granzyme B, and CD107a. This technology holds promise for advancing immune functional assays, offering rapid and accurate on-site analysis of NK cells, a crucial innate immune cell, with its compact and cost-effective optoelectronic setup, especially in the post-COVID-19 era.

Insights into polycrystalline microstructure of blood films with 3D Mueller matrix imaging approach.

This study introduces a novel approach in the realm of liquid biopsies, employing a 3D Mueller-matrix (MM) image reconstruction technique to analyze dehydrated blood smear polycrystalline structures. Our research centers on exploiting the unique optical anisotropy properties of blood proteins, which undergo structural alterations at the quaternary and tertiary levels in the early stages of diseases such as cancer. These alterations manifest as distinct patterns in the polycrystalline microstructure of dried blood droplets, offering a minimally invasive yet highly effective method for early disease detection. We utilized a groundbreaking 3D MM mapping technique, integrated with digital holographic reconstruction, to perform a detailed layer-by-layer analysis of partially depolarizing dry blood smears. This method allows us to extract critical optical anisotropy parameters, enabling the differentiation of blood films from healthy individuals and prostate cancer patients. Our technique uniquely combines polarization-holographic and differential MM methodologies to spatially characterize the 3D polycrystalline structures within blood films. A key advancement in our study is the quantitative evaluation of optical anisotropy maps using statistical moments (first to fourth orders) of linear and circular birefringence and dichroism distributions. This analysis provides a comprehensive characterization of the mean, variance, skewness, and kurtosis of these distributions, crucial for identifying significant differences between healthy and cancerous samples. Our findings demonstrate an exceptional accuracy rate of over 90 % for the early diagnosis and staging of cancer, surpassing existing screening methods. This high level of precision and the non-invasive nature of our technique mark a significant advancement in the field of liquid biopsies. It holds immense potential for revolutionizing cancer diagnosis, early detection, patient stratification, and monitoring, thereby greatly enhancing patient care and treatment outcomes. In conclusion, our study contributes a pioneering technique to the liquid biopsy domain, aligning with the ongoing quest for non-invasive, reliable, and efficient diagnostic methods. It opens new avenues for cancer diagnosis and monitoring, representing a substantial leap forward in personalized medicine and oncology.

A prostate seed implantation robot system based on human-computer interactions: Augmented reality and voice control.

The technology of robot-assisted prostate seed implantation has developed rapidly. However, during the process, there are some problems to be solved, such as non-intuitive visualization effects and complicated robot control. To improve the intelligence and visualization of the operation process, a voice control technology of prostate seed implantation robot in augmented reality environment was proposed. Initially, the MRI image of the prostate was denoised and segmented. The three-dimensional model of prostate and its surrounding tissues was reconstructed by surface rendering technology. Combined with holographic application program, the augmented reality system of prostate seed implantation was built. An improved singular value decomposition three-dimensional registration algorithm based on iterative closest point was proposed, and the results of three-dimensional registration experiments verified that the algorithm could effectively improve the three-dimensional registration accuracy. A fusion algorithm based on spectral subtraction and BP neural network was proposed. The experimental results showed that the average delay of the fusion algorithm was 1.314 s, and the overall response time of the integrated system was 1.5 s. The fusion algorithm could effectively improve the reliability of the voice control system, and the integrated system could meet the responsiveness requirements of prostate seed implantation.

Design and evaluation of an AR-based thermal imaging system for planning reconstructive surgeries.

INTRODUCTION: Thermal imaging can be used for the non-invasive detection of blood vessels of the skin. However, mapping the results to the patient currently lacks user-friendliness. Augmented reality may provide a useful tool to superimpose thermal information on the patient. METHODS: A system to support planning in reconstructive surgery using a thermal camera was designed. The obtained information was superimposed on the physical object using a Microsoft HoloLens. An RGB, depth, and thermal camera were combined to capture a scene of different modalities and reconstruct a virtual scene in real time. To register the different cameras and the AR device, an active calibration target was developed and evaluated. A Vuforia marker was used to register the hologram in the virtual space. The accuracy of the projected hologram was evaluated in a laboratory setting with participants by measuring the error between the physical object and the hologram. RESULTS: The AR-based system was evaluated by 21 participants in a laboratory setting. The mean projection error is 10.3 ± 9.4 mm. The system is able to stream a three-dimensional scene with augmented thermal information in real time at 5 frames per second. The active calibration target can be used independently of the environment. CONCLUSION: The calibration target provides an easy-to-use method for the registration of cameras capturing the visible to long-infrared spectral range. The inside-out tracking of the HoloLens in combination with a Vuforia marker is not accurate enough for the intended clinical use.

Managing Iatrogenic Aortic Dissection: Insight From 3D Holographic Imaging and CT Computational Fluid Dynamic Simulations.

Iatrogenic aortic dissection is a rare but life-threatening complication of coronary artery bypass surgery. We report a case with incidentally detected iatrogenic aortic dissection related to aorta cross-clamping that was successfully managed with watchful follow-up. The decision making was based on 3-dimensional holographic and fluid dynamic analysis guidance.

Molecular fingerprinting of biological nanoparticles with a label-free optofluidic platform.

Label-free detection of multiple analytes in a high-throughput fashion has been one of the long-sought goals in biosensing applications. Yet, for all-optical approaches, interfacing state-of-the-art label-free techniques with microfluidics tools that can process small volumes of sample with high throughput, and with surface chemistry that grants analyte specificity, poses a critical challenge to date. Here, we introduce an optofluidic platform that brings together state-of-the-art digital holography with PDMS microfluidics by using supported lipid bilayers as a surface chemistry building block to integrate both technologies. Specifically, this platform fingerprints heterogeneous biological nanoparticle populations via a multiplexed label-free immunoaffinity assay with single particle sensitivity. First, we characterise the robustness and performance of the platform, and then apply it to profile four distinct ovarian cell-derived extracellular vesicle populations over a panel of surface protein biomarkers, thus developing a unique biomarker fingerprint for each cell line. We foresee that our approach will find many applications where routine and multiplexed characterisation of biological nanoparticles are required.

Assessment of pulmonary vascular anatomy: comparing augmented reality by holograms versus standard CT images/reconstructions using surgical findings as reference standard.

BACKGROUND: We compared computed tomography (CT) images and holograms (HG) to assess the number of arteries of the lung lobes undergoing lobectomy and assessed easiness in interpretation by radiologists and thoracic surgeons with both techniques. METHODS: Patients scheduled for lobectomy for lung cancer were prospectively included and underwent CT for staging. A patient-specific three-dimensional model was generated and visualized in an augmented reality setting. One radiologist and one thoracic surgeon evaluated CT images and holograms to count lobar arteries, having as reference standard the number of arteries recorded at surgery. The easiness of vessel identification was graded according to a Likert scale. Wilcoxon signed-rank test and κ statistics were used. RESULTS: Fifty-two patients were prospectively included. The two doctors detected the same number of arteries in 44/52 images (85%) and in 51/52 holograms (98%). The mean difference between the number of artery branches detected by surgery and CT images was 0.31 ± 0.98, whereas it was 0.09 ± 0.37 between surgery and HGs (p = 0.433). In particular, the mean difference in the number of arteries detected in the upper lobes was 0.67 ± 1.08 between surgery and CT images and 0.17 ± 0.46 between surgery and holograms (p = 0.029). Both radiologist and surgeon showed a higher agreement for holograms (κ = 0.99) than for CT (κ = 0.81) and found holograms easier to evaluate than CTs (p < 0.001). CONCLUSIONS: Augmented reality by holograms is an effective tool for preoperative vascular anatomy assessment of lungs, especially when evaluating the upper lobes, more prone to anatomical variations. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04227444 RELEVANCE STATEMENT: Preoperative evaluation of the lung lobe arteries through augmented reality may help the thoracic surgeons to carefully plan a lobectomy, thus contributing to optimize patients' outcomes. KEY POINTS: • Preoperative assessment of the lung arteries may help surgical planning. • Lung artery detection by augmented reality was more accurate than that by CT images, particularly for the upper lobes. • The assessment of the lung arterial vessels was easier by using holograms than CT images.

Screening for urothelial carcinoma cells in urine based on digital holographic flow cytometry through machine learning and deep learning methods.

The incidence of urothelial carcinoma continues to rise annually, particularly among the elderly. Prompt diagnosis and treatment can significantly enhance patient survival and quality of life. Urine cytology remains a widely-used early screening method for urothelial carcinoma, but it still has limitations including sensitivity, labor-intensive procedures, and elevated cost. In recent developments, microfluidic chip technology offers an effective and efficient approach for clinical urine specimen analysis. Digital holographic microscopy, a form of quantitative phase imaging technology, captures extensive data on the refractive index and thickness of cells. The combination of microfluidic chips and digital holographic microscopy facilitates high-throughput imaging of live cells without staining. In this study, digital holographic flow cytometry was employed to rapidly capture images of diverse cell types present in urine and to reconstruct high-precision quantitative phase images for each cell type. Then, various machine learning algorithms and deep learning models were applied to categorize these cell images, and remarkable accuracy in cancer cell identification was achieved. This research suggests that the integration of digital holographic flow cytometry with artificial intelligence algorithms offers a promising, precise, and convenient approach for early screening of urothelial carcinoma.

[Simulation and Navigation in the Neurosurgical Field Using Three-Dimensional Hholograms by Mixed Reality Devices].

Recently, three-dimensional(3D)holograms from mixed-reality(MR)devices have become available in the medical field. 3D holographic images can provide immersive and intuitive information that has been reported to be very useful for preoperative simulations. Compared with conventional 3D images on a two-dimensional(2D)monitor, 3D holograms offer a higher level of realism, allowing observation of the images anytime and anywhere if the MR device is operational. Even during surgery, surgeons can check realistic 3D holograms in front of them, above the surgical field, without having to turn their heads toward a 2D monitor on the wall. 3D holograms can also be used for neuronavigation if the hologram is tracked to the patient's real head. This method can be defined as 3D augmented reality(AR)navigation, which shows a hologram of a target, such as a tumor or aneurysm, inside the head and brain. In the future, interventions using these techniques with 3D holograms from MR devices are expected to evolve and develop new types of treatments for endoscopic surgery or fluoroscopy-guided endovascular surgery.

The use of mixed reality in the preoperative planning of colorectal surgery: Preliminary experience with a narrative review.

New advanced technologies have recently been developed and preliminarily applied to surgery, including virtual reality (VR), augmented reality (AR) and mixed reality (MR). We retrospectively review all colorectal cases in which we used holographic 3D reconstruction from February 2020 to December 2022. This innovative approach was used to identify vascular anomalies, pinpoint tumor locations, evaluate infiltration into neighboring organs and devise surgical plans for both training and educating trainee assistants. We have also provided a state-of-the-art analysis, briefly highlighting what has been stated by the scientific literature to date. VR facilitates training and anatomical assessments, while AR enhances training and laparoscopic performance evaluations. MR, powered by HoloLens, enriches anatomic recognition, navigation, and visualization. Successful implementation was observed in 10 colorectal cancer cases, showcasing the effectiveness of MR in improving preoperative planning and its intraoperative application. This technology holds significant promise for advancing colorectal surgery by elevating safety and reliability standards.

Comparative oncology chemosensitivity assay for personalized medicine using low-coherence digital holography of dynamic light scattering from cancer biopsies.

Nearly half of cancer patients who receive standard-of-care treatments fail to respond to their first-line chemotherapy, demonstrating the pressing need for improved methods to select personalized cancer therapies. Low-coherence digital holography has the potential to fill this need by performing dynamic contrast OCT on living cancer biopsies treated ex vivo with anti-cancer therapeutics. Fluctuation spectroscopy of dynamic light scattering under conditions of holographic phase stability captures ultra-low Doppler frequency shifts down to 10 mHz caused by light scattering from intracellular motions. In the comparative preclinical/clinical trials presented here, a two-species (human and canine) and two-cancer (esophageal carcinoma and B-cell lymphoma) analysis of spectral phenotypes identifies a set of drug response characteristics that span species and cancer type. Spatial heterogeneity across a centimeter-scale patient biopsy sample is assessed by measuring multiple millimeter-scale sub-samples. Improved predictive performance is achieved for chemoresistance profiling by identifying red-shifted sub-samples that may indicate impaired metabolism and removing them from the prediction analysis. These results show potential for using biodynamic imaging for personalized selection of cancer therapy.

Label-free cell classification in holographic flow cytometry through an unbiased learning strategy.

Nowadays, label-free imaging flow cytometry at the single-cell level is considered the stepforward lab-on-a-chip technology to address challenges in clinical diagnostics, biology, life sciences and healthcare. In this framework, digital holography in microscopy promises to be a powerful imaging modality thanks to its multi-refocusing and label-free quantitative phase imaging capabilities, along with the encoding of the highest information content within the imaged samples. Moreover, the recent achievements of new data analysis tools for cell classification based on deep/machine learning, combined with holographic imaging, are urging these systems toward the effective implementation of point of care devices. However, the generalization capabilities of learning-based models may be limited from biases caused by data obtained from other holographic imaging settings and/or different processing approaches. In this paper, we propose a combination of a Mask R-CNN to detect the cells, a convolutional auto-encoder, used to the image feature extraction and operating on unlabelled data, thus overcoming the bias due to data coming from different experimental settings, and a feedforward neural network for single cell classification, that operates on the above extracted features. We demonstrate the proposed approach in the challenging classification task related to the identification of drug-resistant endometrial cancer cells.

Augmented Reality Holographic Visualization System for Surgery Auxiliary Visualization: Proof of Concept for Surgical Training.

An Augmented Reality (AR) system based on the holographic projection of the relevant anatomic structures is proposed for auxiliary visualization during surgeries. The current two-dimensional visualization systems require the surgeons to mentally extract the associated three-dimensional information during the interventions, which entails risks and complications. This work shows an AR holographic projection system for real-time three-dimensional representation of the relevant surgical information, thus overcoming this problem. As an initial proof of concept, the system is experimentally assessed as potential surgery training tool.Clinical Relevance- This work explores the potential of AR holographic projection systems for intraoperative assistance to the surgical team, starting from its possible use as surgery training and planning tool.