A volumetric display for visual, tactile and audio presentation using acoustic trapping.

Science-fiction movies portray volumetric systems that provide not only visual but also tactile and audible three-dimensional (3D) content. Displays based on swept-volume surfaces1,2, holography3, optophoretics4, plasmonics5 or lenticular lenslets6 can create 3D visual content without the need for glasses or additional instrumentation. However, they are slow, have limited persistence-of-vision capabilities and, most importantly, rely on operating principles that cannot produce tactile and auditive content as well. Here we present the multimodal acoustic trap display (MATD): a levitating volumetric display that can simultaneously deliver visual, auditory and tactile content, using acoustophoresis as the single operating principle. Our system traps a particle acoustically and illuminates it with red, green and blue light to control its colour as it quickly scans the display volume. Using time multiplexing with a secondary trap, amplitude modulation and phase minimization, the MATD delivers simultaneous auditive and tactile content. The system demonstrates particle speeds of up to 8.75 metres per second and 3.75 metres per second in the vertical and horizontal directions, respectively, offering particle manipulation capabilities superior to those of other optical or acoustic approaches demonstrated until now. In addition, our technique offers opportunities for non-contact, high-speed manipulation of matter, with applications in computational fabrication7 and biomedicine8.

Projection of multiple directional images on a volume structure with refractive surfaces.

We present a new refraction-based approach to embed multiple images into a single volume structure rendered on a glass solid (3D crystal). Each of the images can only be revealed when looked at from the certain viewpoint. While configurations of viewing directions in conventional methods are limited, our method can compensate for refractive effects at glass surfaces regardless of the viewing directions and enable the viewing directions to be set more flexibly, even allowing for 180 ∘ opposite projection by leveraging refraction. These unique features are verified with prototyping of 3D crystals projecting multiple grey-scale images and numerical assessments. In addition, we present a color dynamic representation of our method with computer graphics to demonstrate the potential use of our method as a novel information service system.

Improvement of an algorithm for displaying multiple images in one space.

Volumetric displays are attracting attention in fields such as media art and digital signage. In previous research, we developed a method to display multiple images in the same space using a volumetric display. However, because of the nature of the algorithm, the images could not be displayed when they contained a pure black image (in which all the pixel values are "0"). In the current study, we present a revised algorithm that can display such images. Therefore, a wider range of images can be displayed in the same space. Image quality evaluation using structural similarity shows that the proposed algorithm yields images that are superior or equivalent to those of the previous algorithm.

Special-purpose computer HORN-8 for phase-type electro-holography.

Electro-holography is a promising display technology that can reconstruct a photorealistic three-dimensional (3D) movie; however, it is yet to be realized practically owing to the need for enormous calculation power. A special-purpose computer for electro-holography, namely HORN, has been studied for over 20 years as a means to solve this problem. The latest version of HORN, HORN-8, was developed using field programmable gate array (FPGA) technology. Initially, a circuit for amplitude-type electro-holography was implemented in HORN-8; however, implementation of phase-type electro-holography has remained an issue. In this paper, the development of new version of HORN-8 and its cluster system, which achieved a real-time reconstruction of a 3D movie with point clouds comprised of 32,000 points for phase-type electro-holography, was reported.

Efficient method for fabricating a directional volumetric display using strings displaying multiple images.

In this study, we describe the fabrication of a high-resolution directional volumetric display that can display multiple images in different directions. The display designs can be used to show animations using strings; however, improving the resolution of such displays is difficult. Previously, the arrangement of strings has only been determined experimentally, making fabrication of volumetric displays a challenge. The goal of the present study is to improve resolution using simulations and to determine the arrangement of strings under three constraints. This simplified the fabrication of a directional volumetric display with 345 strings, which can display two different 20×20 pixel images in two different directions. A large high-resolution directional volumetric display can be fabricated using the proposed method. The string-type display has high artistic potential and is expected to find applications in the amusement and entertainment fields.

Convolutional neural network-based data page classification for holographic memory.

We propose a deep-learning-based classification of data pages used in holographic memory. We numerically investigated the classification performance of a conventional multilayer perceptron (MLP) and a deep neural network, under the condition that reconstructed page data are contaminated by some noise and are randomly laterally shifted. When data pages are randomly laterally shifted, the MLP was found to have a classification accuracy of 93.02%, whereas the deep neural network was able to classify data pages at an accuracy of 99.98%. The accuracy of the deep neural network is 2 orders of magnitude better than the MLP.

Autoencoder-based holographic image restoration.

We propose a holographic image restoration method using an autoencoder, which is an artificial neural network. Because holographic reconstructed images are often contaminated by direct light, conjugate light, and speckle noise, the discrimination of reconstructed images may be difficult. In this paper, we demonstrate the restoration of reconstructed images from holograms that record page data in holographic memory and quick response codes by using the proposed method.

Holographic microinformation hiding.

We propose a holographic microinformation hiding scheme in which the embedding information to be embedded is small and imperceptible to the human eyes. This scheme converts the embedding information into a complex amplitude via scaled diffraction. The complex amplitude of the reduced embedding information is added to the complex amplitude of the host image, followed by conversion to a hologram. The reduced embedded information is inconspicuous from the hologram during the reconstruction process; however, the reduction leads to the degradation of the embedded image quality. Therefore, to improve the quality of the embedded image quality, we employ iterative optimization and the time averaging effect of multiple holograms.

Image quality improvement for a 3D structure exhibiting multiple 2D patterns and its implementation.

A three-dimensional (3D) structure designed by our proposed algorithm can simultaneously exhibit multiple two-dimensional patterns. The 3D structure provides multiple patterns having directional characteristics by distributing the effects of the artefacts. In this study, we proposed an iterative algorithm to improve the image quality of the exhibited patterns and have verified the effectiveness of the proposed algorithm using numerical simulations. Moreover, we fabricated different 3D glass structures (an octagonal prism, a cube and a sphere) using the proposed algorithm. All 3D structures exhibit four patterns, and different patterns can be observed depending on the viewing direction.

Color computer-generated hologram generation using the random phase-free method and color space conversion.

We propose two calculation methods of generating color computer-generated holograms (CGHs) with the random phase-free method and color space conversion in order to improve the image quality and accelerate the calculation. The random phase-free method improves the image quality in monochrome CGH, but it is not performed in color CGH. We first aimed to improve the image quality of color CGH using the random phase-free method and then to accelerate the color CGH generation with a combination of the random phase-free method and color space conversion method, which accelerates the color CGH calculation due to down-sampling of the color components converted by color space conversion. To overcome the problem of image quality degradation that occurs due to the down-sampling of random phases, the combination of the random phase-free method and color space conversion method improves the quality of reconstructed images and accelerates the color CGH calculation. We demonstrated the effectiveness of the proposed method in simulation, and in this paper discuss its application to lensless zoomable holographic projection.

Acceleration of computer-generated holograms using tilted wavefront recording plane method.

Computer Generated Holograms (CGH) are generated on computers; however, a great deal of computational power is required because the quality of the image is proportional to the number of point light sources of a 3D object. The Wavefront Recording Plane (WRP) method is an algorithm that enables reduction of the amount of calculations required. However, the WRP method also has a defect; it is not effective in the case of a 3D object with a deep structure. In this study, we propose two improved WRP methods: "Least Square Tilted WRP method" and "RANSAC Multi-Tilted WRP method."

Random phase-free kinoform for large objects.

We propose a random phase-free kinoform for large objects. When not using the random phase in kinoform calculation, the reconstructed images from the kinoform are heavy degraded, like edge-only preserved images. In addition, the kinoform cannot record an entire object that exceeds the kinoform size because the object light does not widely spread. In order to avoid this degradation and to widely spread the object light, the random phase is applied to the kinoform calculation; however, the reconstructed image is contaminated by speckle noise. In this paper, we overcome this problem by using our random phase-free method and error diffusion method.

Lensless zoomable holographic projection using scaled Fresnel diffraction.

Projectors require a zoom function. This function is generally realized using a zoom lens module composed of many lenses and mechanical parts; however, using a zoom lens module increases the system size and cost, and requires manual operation of the module. Holographic projection is an attractive technique because it inherently requires no lenses, reconstructs images with high contrast and reconstructs color images with one spatial light modulator. In this paper, we demonstrate a lensless zoomable holographic projection. Without using a zoom lens module, this holographic projection realizes the zoom function using a numerical method, called scaled Fresnel diffraction which can calculate diffraction at different sampling rates on a projected image and hologram.

Nonuniform sampled scalar diffraction calculation using nonuniform fast Fourier transform.

Scalar diffraction calculations, such as the angular spectrum method (ASM) and Fresnel diffraction, are widely used in the research fields of optics, x rays, electron beams, and ultrasonics. It is possible to accelerate the calculation using fast Fourier transform (FFT); unfortunately, acceleration of the calculation of nonuniform sampled planes is limited due to the property of the FFT that imposes uniform sampling. In addition, it gives rise to wasteful sampling data if we calculate a plane having locally low and high spatial frequencies. In this Letter, we developed nonuniform sampled ASM and Fresnel diffraction to improve the problem using the nonuniform FFT.

Machine Learning in Predicting Printable Biomaterial Formulations for Direct Ink Writing.

Three-dimensional (3D) printing is emerging as a transformative technology for biomedical engineering. The 3D printed product can be patient-specific by allowing customizability and direct control of the architecture. The trial-and-error approach currently used for developing the composition of printable inks is time- and resource-consuming due to the increasing number of variables requiring expert knowledge. Artificial intelligence has the potential to reshape the ink development process by forming a predictive model for printability from experimental data. In this paper, we constructed machine learning (ML) algorithms including decision tree, random forest (RF), and deep learning (DL) to predict the printability of biomaterials. A total of 210 formulations including 16 different bioactive and smart materials and 4 solvents were 3D printed, and their printability was assessed. All ML methods were able to learn and predict the printability of a variety of inks based on their biomaterial formulations. In particular, the RF algorithm has achieved the highest accuracy (88.1%), precision (90.6%), and F1 score (87.0%), indicating the best overall performance out of the 3 algorithms, while DL has the highest recall (87.3%). Furthermore, the ML algorithms have predicted the printability window of biomaterials to guide the ink development. The printability map generated with DL has finer granularity than other algorithms. ML has proven to be an effective and novel strategy for developing biomaterial formulations with desired 3D printability for biomedical engineering applications.

High-speed acoustic holography with arbitrary scattering objects.

Recent advances in high-speed acoustic holography have enabled levitation-based volumetric displays with tactile and audio sensations. However, current approaches do not compute sound scattering of objects' surfaces; thus, any physical object inside can distort the sound field. Here, we present a fast computational technique that allows high-speed multipoint levitation even with arbitrary sound-scattering surfaces and demonstrate a volumetric display that works in the presence of any physical object. Our technique has a two-step scattering model and a simplified levitation solver, which together can achieve more than 10,000 updates per second to create volumetric images above and below static sound-scattering objects. The model estimates transducer contributions in real time by reformulating the boundary element method for acoustic holography, and the solver creates multiple levitation traps. We explain how our technique achieves its speed with minimum loss in the trap quality and illustrate how it brings digital and physical content together by demonstrating mixed-reality interactive applications.

Inkjet printing-based volumetric display projecting multiple full-colour 2D patterns.

In this study, a method to construct a full-colour volumetric display is presented using a commercially available inkjet printer. Photoreactive luminescence materials are minutely and automatically printed as the volume elements, and volumetric displays are constructed with high resolution using easy-to-fabricate means that exploit inkjet printing technologies. The results experimentally demonstrate the first prototype of an inkjet printing-based volumetric display composed of multiple layers of transparent films that yield a full-colour three-dimensional (3D) image. Moreover, we propose a design algorithm with 3D structures that provide multiple different 2D full-colour patterns when viewed from different directions and experimentally demonstrate prototypes. It is considered that these types of 3D volumetric structures and their fabrication methods based on widely deployed existing printing technologies can be utilised as novel information display devices and systems, including digital signage, media art, entertainment and security.

Optical Addressing of Multi-Colour Photochromic Material Mixture for Volumetric Display.

This is the first study to demonstrate that colour transformations in the volume of a photochromic material (PM) are induced at the intersections of two control light channels, one controlling PM colouration and the other controlling decolouration. Thus, PM colouration is induced by position selectivity, and therefore, a dynamic volumetric display may be realised using these two control lights. Moreover, a mixture of multiple PM types with different absorption properties exhibits different colours depending on the control light spectrum. Particularly, the spectrum management of the control light allows colour-selective colouration besides position selectivity. Therefore, a PM-based, full-colour volumetric display is realised. We experimentally construct a mixture of two PM types and validate the operating principles of such a volumetric display system. Our system is constructed simply by mixing multiple PM types; therefore, the display hardware structure is extremely simple, and the minimum size of a volume element can be as small as the size of a molecule. Volumetric displays can provide natural three-dimensional (3D) perception; therefore, the potential uses of our system include high-definition 3D visualisation for medical applications, architectural design, human-computer interactions, advertising, and entertainment.

Design, implementation and characterization of a quantum-dot-based volumetric display.

In this study, we propose and experimentally demonstrate a volumetric display system based on quantum dots (QDs) embedded in a polymer substrate. Unlike conventional volumetric displays, our system does not require electrical wiring; thus, the heretofore unavoidable issue of occlusion is resolved because irradiation by external light supplies the energy to the light-emitting voxels formed by the QDs. By exploiting the intrinsic attributes of the QDs, the system offers ultrahigh definition and a wide range of colours for volumetric displays. In this paper, we discuss the design, implementation and characterization of the proposed volumetric display's first prototype. We developed an 8 × 8 × 8 display comprising two types of QDs. This display provides multicolour three-type two-dimensional patterns when viewed from different angles. The QD-based volumetric display provides a new way to represent images and could be applied in leisure and advertising industries, among others.

Three-dimensional volume containing multiple two-dimensional information patterns.

We have developed an algorithm for recording multiple gradated two-dimensional projection patterns in a single three-dimensional object. When a single pattern is observed, information from the other patterns can be treated as background noise. The proposed algorithm has two important features: the number of patterns that can be recorded is theoretically infinite and no meaningful information can be seen outside of the projection directions. We confirmed the effectiveness of the proposed algorithm by performing numerical simulations of two laser crystals: an octagonal prism that contained four patterns in four projection directions and a dodecahedron that contained six patterns in six directions. We also fabricated and demonstrated an actual prototype laser crystal from a glass cube engraved by a laser beam. This algorithm has applications in various fields, including media art, digital signage, and encryption technology.