Learned holographic light transport: invited.

Computer-generated holography algorithms often fall short in matching simulations with results from a physical holographic display. Our work addresses this mismatch by learning the holographic light transport in holographic displays. Using a camera and a holographic display, we capture the image reconstructions of optimized holograms that rely on ideal simulations to generate a dataset. Inspired by the ideal simulations, we learn a complex-valued convolution kernel that can propagate given holograms to captured photographs in our dataset. Our method can dramatically improve simulation accuracy and image quality in holographic displays while paving the way for physically informed learning approaches.

Light-efficient augmented reality display with steerable eyebox.

We present a novel head-mounted display setup that uses the pinhole imaging principle coupled with a low-latency dynamic pupil follower. A transmissive LCD is illuminated by a single LED backlight. LED illumination is focused onto the viewer's pupil to form an eyebox smaller than the average human pupil, thereby creating a pinhole display effect where objects at all distances appear in focus. Since nearly all the light is directed to the viewer's pupil, a single low-power LED for each primary color with 0.42 lumens total output is sufficient to create a bright and full-color display of 360 cd/m2 luminance. In order to follow the viewer's pupil, the eyebox needs to be steerable. We achieved a dynamic eyebox using an array of LEDs that is coupled with a real-time pupil tracker. The entire system is operated at 11 msec motion-to-photon latency, which meets the demanding requirements of the real-time pupil follower system. Experimental results effectively demonstrated our head-mounted pinhole display with 37° FOV and very high light efficiency, equipped with a pupil follower with low motion-to-photon latency.

Micro-mirror-array based off-axis flat lens for near-eye displays.

We developed an off-axis diffractive lens using a micro-mirror array on a flat substrate. MMA creates an on-axis converging beam from a 45 degrees off-axis diverging illumination beam and functions similar to a large and bulky elliptical mirror. The array consists of individual micro-mirrors with normal directions that vary across the component. The size, normal direction and the center height of each micro-mirror are optimized to achieve a phase matching condition so that the smallest focal spot size is achieved at the design wavelength. Design can also be optimized for full color applications using a synthetic design wavelength. A sample MMA of size 3 mm by 5 mm is fabricated using grayscale lithography. The designed MMA is used to illuminate a computer-generated hologram in a near-eye display system. Experimental results verify the premises of the designed component.

Integrated 3D display and imaging using dual purpose passive screen and head-mounted projectors and camera.

We propose an integrated 3D display and imaging system using a head-mounted device and a special dual-purpose passive screen that can simultaneously facilitate 3D display and imaging. The screen is mainly composed of two optical layers, the first layer is a projection surface, which are the finely patterned retro-reflective microspheres that provide high optical gain when illuminated with head-mounted projectors. The second layer is an imaging surface made up of an array of curved mirrors, which form the perspective views of the scene captured by a head-mounted camera. The display and imaging operation are separated by performing polarization multiplexing. The demonstrated prototype system consists of a head-worn unit having a pair of 15 lumen pico-projectors and a 24MP camera, and an in-house designed and fabricated 30cm × 24cm screen. The screen provides bright display using 25% filled retro-reflective microspheres and 20 different perspective views of the user/scene using 5 × 4 array of convex mirrors. The real-time implementation is demonstrated by displaying stereo-3D content providing high brightness (up to 240 cd/m2) and low crosstalk (<4%), while 3D image capture is demonstrated by performing the computational reconstruction of the discrete free-viewpoint stereo pair displayed on a desktop or virtual reality display. Furthermore, the capture quality is determined by measuring the imaging MTF of the captured views and the capture light efficiency is calculated by considering the loss in transmitted light at each interface. Further developments in microfabrication and computational optics can present the proposed system as a unique mobile platform for immersive human-computer interaction of the future.

Scanning fiber microdisplay: design, implementation, and comparison to MEMS mirror-based scanning displays.

In this study, we propose a compact, lightweight scanning fiber microdisplay towards virtual and augmented reality applications. Our design that is tailored as a head-worn-display simply consists of a four-quadrant piezoelectric tube actuator through which a fiber optics cable is extended and actuated, and a reflective (or semi-reflective) ellipsoidal surface that relays the moving tip of the fiber onto the viewer's retina. The proposed display, offers significant advantages in terms of architectural simplicity, form-factor, fabrication complexity and cost over other fiber scanner and MEMS mirror counterparts towards practical realization. We demonstrate the display of various patterns with ∼VGA resolution and further provide analytical formulas for mechanical and optical constraints to compare the performance of the proposed scanning fiber microdisplay with that of MEMS mirror-based microdisplays. Also we discuss the road steps towards improving the performance of the proposed scanning fiber microdisplay to high-definition video formats (such as HD1440), which is beyond what has been achieved by MEMS mirror based laser scanning displays.

Light-efficient augmented reality 3D display using highly transparent retro-reflective screen.

We propose and demonstrate a light-efficient 3D display using a highly transparent desktop-sized augmented reality screen. The display consists of a specially designed transparent retro-reflective screen and a pair of low-power pico-projectors positioned close to the viewer's eyes to provide stereo views. The transfer screen is an optically clear sheet partially patterned with retro-reflective microspheres for high optical gain. The retro-reflective material buried in the screen reflects incident light back towards the projectors with a narrow scattering angle and facilitates the viewer to perceive a very bright content. The tabletop prototype mainly consists of an in-house fabricated large augmented reality (AR) screen (60 cm×40 cm) and a pair of laser-scanning 30 lumen pico-projectors. The display is tested for different viewing configurations, and different display parameters, such as retro-reflective coefficient, eye-box size, polarization maintainability, stereo crosstalk, and brightness, are examined. The AR prototype display provides 75% optical transparency, exceptional brightness (up to 1000 cd/m2 when viewed through beam splitters and 350 cd/m2 with bare eyes), and negligible crosstalk in 3D mode (<5% and <1% when viewed through beam splitters and polarizers, respectively) for the working distance of up to 2 m.

Design, fabrication and characterization of transparent retro-reflective screen.

A transparent retro-reflective screen, which can be used as head-up-display (HUD) or a see-through screen for head mounted projection displays (HMPD) is proposed. The high optical gain of screen enables the use of low power projectors to produce very bright content. The screen assembly is based on retro-reflective microspheres, patterned on an optically clear substrate using steel stencil as a shadow mask. The incident light is retro-reflected as a narrow angular cone to create an eyebox for the viewer. The optical gain and transparency of screen is varied by changing the fill factor of the mask. The optical design and fabrication of the screen is presented. The retro-reflective and transmission characteristics of screen are evaluated. The impact of fill factor on screen luminance and transparency is studied. The screen provides high luminance (up to 280cd/m2 with 50% transparency) from about 40cm to >3m when used with a low power (15 lumen) mobile projector. Unlike regular diffusers, luminance remains nearly constant with projection distance. Furthermore, the screen offers prominent see-through capability with small degradation in modulation transfer function for transmitted light. For a particular camera and imaging configuration, MTF10 (10% cutoff) for 50% transparent screen is reduced from 37 cyc/deg to 30 cyc/deg when screen is inserted at an intermediate distance.

Non-iterative phase hologram computation for low speckle holographic image projection.

Phase-only spatial light modulators (SLMs) are widely used in holographic display applications, including holographic image projection (HIP). Most phase computer generated hologram (CGH) calculation algorithms have an iterative structure with a high computational load, and also are prone to speckle noise, as a result of the random phase terms applied on the desired images to mitigate the encoding noise. In this paper, we present a non-iterative algorithm, where simple Discrete Fourier Transform (DFT) relations are exploited to compute phase CGHs that exactly control half of the desired image samples (those on even - or odd - indexed rows - or columns) via a single Fast Fourier Transform (FFT) and trivial arithmetic operations. The encoding noise appearing on the uncontrolled half of the image samples is reduced by the application of structured, non-random initial phase terms so that speckle noise is also kept low. High quality reconstructions are obtained under temporal averaging of several SLM frames. Interlaced video within half of the addressable image area is readily deliverable without frame rate division. Our algorithm provides about 6X and 20X reduction in computational cost compared to IFTA and FIDOC algorithms, respectively. Simulations and experiments verify that the algorithm constitutes a promising option for real-time computation of phase CGHs.

Super stereoscopy technique for comfortable and realistic 3D displays.

Two well-known problems of stereoscopic displays are the accommodation-convergence conflict and the lack of natural blur for defocused objects. We present a new technique that we name Super Stereoscopy (SS3D) to provide a convenient solution to these problems. Regular stereoscopic glasses are replaced by SS3D glasses which deliver at least two parallax images per eye through pinholes equipped with light selective filters. The pinholes generate blur-free retinal images so as to enable correct accommodation, while the delivery of multiple parallax images per eye creates an approximate blur effect for defocused objects. Experiments performed with cameras and human viewers indicate that the technique works as desired. In case two, pinholes equipped with color filters per eye are used; the technique can be used on a regular stereoscopic display by only uploading a new content, without requiring any change in display hardware, driver, or frame rate. Apart from some tolerable loss in display brightness and decrease in natural spatial resolution limit of the eye because of pinholes, the technique is quite promising for comfortable and realistic 3D vision, especially enabling the display of close objects that are not possible to display and comfortably view on regular 3DTV and cinema.

Microcantilever based disposable viscosity sensor for serum and blood plasma measurements.

This paper proposes a novel method for measuring blood plasma and serum viscosity with a microcantilever-based MEMS sensor. MEMS cantilevers are made of electroplated nickel and actuated remotely with magnetic field using an electro-coil. Real-time monitoring of cantilever resonant frequency is performed remotely using diffraction gratings fabricated at the tip of the dynamic cantilevers. Only few nanometer cantilever deflection is sufficient due to interferometric sensitivity of the readout. The resonant frequency of the cantilever is tracked with a phase lock loop (PLL) control circuit. The viscosities of liquid samples are obtained through the measurement of the cantilever's frequency change with respect to a reference measurement taken within a liquid of known viscosity. We performed measurements with glycerol solutions at different temperatures and validated the repeatability of the system by comparing with a reference commercial viscometer. Experimental results are compared with the theoretical predictions based on Sader's theory and agreed reasonably well. Afterwards viscosities of different Fetal Bovine Serum and Bovine Serum Albumin mixtures are measured both at 23°C and 37°C, body temperature. Finally the viscosities of human blood plasma samples taken from healthy donors are measured. The proposed method is capable of measuring viscosities from 0.86 cP to 3.02 cP, which covers human blood plasma viscosity range, with a resolution better than 0.04 cP. The sample volume requirement is less than 150 µl and can be reduced significantly with optimized cartridge design. Both the actuation and sensing are carried out remotely, which allows for disposable sensor cartridges.

High-speed broadband FTIR system using MEMS.

Current Fourier transform infrared spectroscopy (FTIR) systems have very good spectral resolution, but are bulky, sensitive to vibrations, and slow. We developed a new FTIR system using a microelectromechanical system (MEMS)-based lamellar grating interferometer that is fast, compact, and achromatic (i.e., does not require a beam splitter). The MEMS device has >10 mm2 active surface area, up to ±325 µm mechanical displacement, and a 343 Hz resonant operation frequency. The system uses a 5 MHz bandwidth custom infrared (IR) detector and a small emission area custom blackbody source to achieve fast interferogram acquisition and compact form factor. Effects of lamellar grating period, detector size, laser reference, apodization, and averaging of data on the spectral resolution are discussed. The measurement time ranges from 1.5 to 100 ms depending on the averaging time. In the target range of 2.5-16 µm (625-4000 cm-1) a spectral resolution of 15-20 cm-1 is demonstrated. The measurements are shown to be stable over a long time.

A Biofeedback Based Virtual Reality Game for Pediatric Population (BioVirtualPed): A Feasibility Trial.

OBJECTIVE: This trial aims to assess the acceptability, feasibility, and safety of BioVirtualPed, a biofeedback-based virtual reality (VR) game designed to reduce pain, anxiety, and fear in children undergoing medical procedures. METHODS: An Oculus Quest 2 headset was used in the VR experience, respiratory data was captured using an ADXL354 accelerometer, and these data were integrated into the game with ArdunioUno software. The sample of this study consisted of 15 pediatric oncology patients aged 6 to 12 years between July and August 2023. BioVirtualPed's acceptability, feasibility, and safety were evaluated through child and expert feedback, alongside metrics including the System Usability Scale, Wong-Baker Pain Rating Scale, Child Fear Scale, Child Anxiety Scale-Status, Satisfaction Scoring, and various feasibility and safety parameters. RESULTS: Regarding the acceptability, the expert evaluation showed a mean score of 122.5 ± 3.53, indicating high usability for the system. All children provided positive feedback, and both children and their mothers reported high satisfaction with using BioVirtualPed. The BioVirtualPed was feasible for reducing children's pain, fear, and anxiety levels. All the children complied with the game, and no one withdrew from the trial. BioVirtualPed did not cause symptoms of dizziness, vomiting, or nausea in children and was found to be safe for children. CONCLUSION: The findings showed that BioVirtualPed meets the following criteria: feasibility, user satisfaction, acceptability, and safety. It is a valuable tool to improve children's experience undergoing port catheter needle insertion procedures. IMPLICATION FOR NURSING PRACTICE: Integration of VR interventions with BioVirtualPed into routine nursing care practices has the potential to effectively manage the pain, anxiety, and fear experienced by children undergoing medical procedures. The safety, feasibility, and acceptability results are promising for further research and integration into pediatric healthcare practice.

Multi-view autostereoscopic projection display using rotating screen.

A new technique for multi-view autostereoscopic projection display is proposed, and demonstrated. The technique uses two mobile projectors, a rotating retro-reflective diffuser screen, and a head-tracking camera. As two dynamic viewing slits are created at the viewer's position, the slits can track the position of the eyes by rotating the screen. The display allows a viewer to move approximately 700 mm along the horizontal axis, and 500 mm along the vertical axis with an average crosstalk below 5 %. Two screen prototypes with different diffusers have been tried, and they provide luminance levels of 60 Cd/m2, and 160 Cd/m2 within the viewing field.

Transmission characteristics of a bidirectional transparent screen based on reflective microlenses.

A microlens array (MLA) based see-through, front projection screen, which can be used in direct projection head-up displays (HUD), color teleprompters and bidirectional interactive smart windows, is evaluated for various performance metrics in transmission mode. The screen structure consists of a partially reflective coated MLA buried between refractive-index-matched layers of epoxy as reported in Ref [1]. The reflected light is expanded by the MLA to create an eyebox for the user. The brightness gain of the screen can be varied by changing the numerical aperture of the microlenses. Thus, using high gain designs, a low-power projector coupled with the screen can produce high-brightness and even 3D images as the polarization is maintained at the screen. The impact of the partially reflective coatings on the transmitted light in terms of resolution and modulation transfer function associated with the screen is studied. A condition similar to the Rayleigh criteria for diffraction-limited imaging is discussed for the microlens arrays and the associated coating layers. The optical path difference between the light transmitted from the center and the edges of each microlens caused by the reflective layer coatings should not exceed λ/4. Furthermore, the crosstalk between the front and rear projected images is found to be less than 1.3%.

Dynamic exit pupil trackers for autostereoscopic displays.

This paper describes the first demonstrations of two dynamic exit pupil (DEP) tracker techniques for autostereoscopic displays. The first DEP tracker forms an exit pupil pair for a single viewer in a defined space with low intraocular crosstalk using a pair of moving shutter glasses located within the optical system. A display prototype using the first DEP tracker is constructed from a pair of laser projectors, pupil-forming optics, moving shutter glasses at an intermediate pupil plane, an image relay lens, and a Gabor superlens based viewing screen. The left and right eye images are presented time-sequentially to a single viewer and seen as a 3D image without wearing glasses and allows the viewer to move within a region of 40 cm × 20 cm in the lateral plane, and 30 cm along the axial axis. The second DEP optics can move the exit pupil location dynamically in a much larger 3D space by using a custom spatial light modulator (SLM) forming an array of shutters. Simultaneous control of multiple exit pupils in both lateral and axial axes is demonstrated for the first time and provides a viewing volume with an axial extent of 0.6-3 m from the screen and within a lateral viewing angle of ± 20° for multiple viewers. This system has acceptable crosstalk (< 5%) between the stereo image pairs. In this novel version of the display the optical system is used as an advanced dynamic backlight for a liquid crystal display (LCD). This has advantages in terms of overall display size as there is no requirement for an intermediate image, and in image quality. This system has acceptable crosstalk (< 5%) between the stereo image pairs.

Microlens array-based high-gain screen design for direct projection head-up displays.

Head-up display (HUD) systems have been used in recent car models to provide essential information to the drivers while keeping their eyes on the road. Virtual image HUD systems have been the preferred method, but they have the drawback of requiring a large volume of space in order to accommodate the relay optics that creates the virtual image. This is especially significant as the desired field of view increases. Direct projection HUD systems have been developed with a separate stand-alone microlens array (MLA)-based transparent screen on the dashboard, offering a compact solution. In this paper, we propose a direct projection HUD system based on a unique, windshield-embedded see-through screen that uses minimal space under the dashboard, offering an elegant and compact solution to the HUD problem. The screen is based on MLAs with varying surface normal angles such that the light from the projector is directed to the viewer's eyes from all positions across the field of view. Varying tilts provide an efficient relay and high brightness even with a low-lumen output projector. The calculated screen gain is about 69 and the eyebox area is about 30 cm×30 cm.

Dynamic accommodation measurement using Purkinje reflections and machine learning.

Quantifying eye movement is important for diagnosing various neurological and ocular diseases as well as AR/VR displays. We developed a simple setup for real-time dynamic gaze tracking and accommodation measurements based on Purkinje reflections, which are the reflections from front and back surfaces of the cornea and the eye lens. We used an accurate eye model in ZEMAX to simulate the Purkinje reflection positions at different focus distances of the eye, which matched the experimental data. A neural network was trained to simultaneously predict vergence and accommodation using data collected from 9 subjects. We demonstrated that the use of Purkinje reflection coordinates in machine learning resulted in precise estimation. The proposed system accurately predicted the accommodation with an accuracy better than 0.22 D using subject's own data and 0.40 D using other subjects' data with two-point calibration in tests performed with 9 subjects in our setup.

Fourier optics analysis of grating sensors with tilt errors.

Dynamic diffraction gratings can be microfabricated with precision and offer extremely sensitive displacement measurements and light intensity modulation. The effect of pure translation of the moving part of the grating on diffracted order intensities is well known. This study focuses on the parameters that limit the intensity and the contrast of the interference. The effects of grating duty cycle, mirror reflectivities, sensor tilt and detector size are investigated using Fourier optics theory and Gaussian beam optics. Analytical findings reveal that fringe visibility becomes <0.3 when the optical path variation exceeds half the wavelength within the grating interferometer. The fringe visibility can be compensated by monitoring the interfering portion of the diffracted order light only through detector size reduction in the expense of optical power. Experiments were conducted with a grating interferometer that resulted in an eightfold increase in fringe visibility with reduced detector size, which is in agreement with theory. Findings show that diffraction grating readout principle is not limited to translating sensors but also can be used for sensors with tilt or other deflection modes.

Sensitivity enhancement of grating interferometer based two-dimensional sensor arrays using two-wavelength readout.

Diffraction gratings integrated with microelectromechanical systems (MEMS) sensors offer displacement measurements with subnanometer sensitivity. However, the sensitivity of the interferometric readout may drop significantly based on the gap between the grating and the reference surface. A two-wavelength (2-λ) readout method was previously tested using a single MEMS sensor for illustrating increased displacement measurement capability. This work demonstrates sensitivity enhancement on a sensor array with large scale parallelization (~20,000 sensors). The statistical representation, which is developed to model sensitivity enhancement within a grating based sensor array, is supported by experimental results using a thermal sensor array. In the experiments, two lasers at different wavelengths (633 and 650 nm) illuminate the thermal sensor array from the backside, time-sequentially. The diffracted first order light from the array is imaged onto a single CCD camera. The target scene is reconstructed by observing the change in the first diffracted order diffraction intensity for both wavelengths. Merging of the data from two measurements with two lasers was performed by taking the larger of the two CCD measurements with respect to the reference image for each sensor. ~30% increase in the average sensitivity was demonstrated for a 160×120 pixel IR sensor array. Proposed architecture is also applicable to a variety of sensing applications, such as parallel biosensing and atomic force microscopy, for improved displacement measurements and enhanced sensitivity.

Applications of augmented reality in ophthalmology [Invited].

Throughout the last decade, augmented reality (AR) head-mounted displays (HMDs) have gradually become a substantial part of modern life, with increasing applications ranging from gaming and driver assistance to medical training. Owing to the tremendous progress in miniaturized displays, cameras, and sensors, HMDs are now used for the diagnosis, treatment, and follow-up of several eye diseases. In this review, we discuss the current state-of-the-art as well as potential uses of AR in ophthalmology. This review includes the following topics: (i) underlying optical technologies, displays and trackers, holography, and adaptive optics; (ii) accommodation, 3D vision, and related problems such as presbyopia, amblyopia, strabismus, and refractive errors; (iii) AR technologies in lens and corneal disorders, in particular cataract and keratoconus; (iv) AR technologies in retinal disorders including age-related macular degeneration (AMD), glaucoma, color blindness, and vision simulators developed for other types of low-vision patients.