Controlling Visually Guided Behavior by Holographic Recalling of Cortical Ensembles.

Neurons in cortical circuits are often coactivated as ensembles, yet it is unclear whether ensembles play a functional role in behavior. Some ensemble neurons have pattern completion properties, triggering the entire ensemble when activated. Using two-photon holographic optogenetics in mouse primary visual cortex, we tested whether recalling ensembles by activating pattern completion neurons alters behavioral performance in a visual task. Disruption of behaviorally relevant ensembles by activation of non-selective neurons decreased performance, whereas activation of only two pattern completion neurons from behaviorally relevant ensembles improved performance, by reliably recalling the whole ensemble. Also, inappropriate behavioral choices were evoked by the mistaken activation of behaviorally relevant ensembles. Finally, in absence of visual stimuli, optogenetic activation of two pattern completion neurons could trigger behaviorally relevant ensembles and correct behavioral responses. Our results demonstrate a causal role of neuronal ensembles in a visually guided behavior and suggest that ensembles implement internal representations of perceptual states.

Connectomic reconstruction of a female Drosophila ventral nerve cord.

A deep understanding of how the brain controls behaviour requires mapping neural circuits down to the muscles that they control. Here, we apply automated tools to segment neurons and identify synapses in an electron microscopy dataset of an adult female Drosophila melanogaster ventral nerve cord (VNC)1, which functions like the vertebrate spinal cord to sense and control the body. We find that the fly VNC contains roughly 45 million synapses and 14,600 neuronal cell bodies. To interpret the output of the connectome, we mapped the muscle targets of leg and wing motor neurons using genetic driver lines2 and X-ray holographic nanotomography3. With this motor neuron atlas, we identified neural circuits that coordinate leg and wing movements during take-off. We provide the reconstruction of VNC circuits, the motor neuron atlas and tools for programmatic and interactive access as resources to support experimental and theoretical studies of how the nervous system controls behaviour.

Towards real-time photorealistic 3D holography with deep neural networks.

The ability to present three-dimensional (3D) scenes with continuous depth sensation has a profound impact on virtual and augmented reality, human-computer interaction, education and training. Computer-generated holography (CGH) enables high-spatio-angular-resolution 3D projection via numerical simulation of diffraction and interference1. Yet, existing physically based methods fail to produce holograms with both per-pixel focal control and accurate occlusion2,3. The computationally taxing Fresnel diffraction simulation further places an explicit trade-off between image quality and runtime, making dynamic holography impractical4. Here we demonstrate a deep-learning-based CGH pipeline capable of synthesizing a photorealistic colour 3D hologram from a single RGB-depth image in real time. Our convolutional neural network (CNN) is extremely memory efficient (below 620 kilobytes) and runs at 60 hertz for a resolution of 1,920 × 1,080 pixels on a single consumer-grade graphics processing unit. Leveraging low-power on-device artificial intelligence acceleration chips, our CNN also runs interactively on mobile (iPhone 11 Pro at 1.1 hertz) and edge (Google Edge TPU at 2.0 hertz) devices, promising real-time performance in future-generation virtual and augmented-reality mobile headsets. We enable this pipeline by introducing a large-scale CGH dataset (MIT-CGH-4K) with 4,000 pairs of RGB-depth images and corresponding 3D holograms. Our CNN is trained with differentiable wave-based loss functions5 and physically approximates Fresnel diffraction. With an anti-aliasing phase-only encoding method, we experimentally demonstrate speckle-free, natural-looking, high-resolution 3D holograms. Our learning-based approach and the Fresnel hologram dataset will help to unlock the full potential of holography and enable applications in metasurface design6,7, optical and acoustic tweezer-based microscopic manipulation8-10, holographic microscopy11 and single-exposure volumetric 3D printing12,13.

Airy-beam holographic sonogenetics for advancing neuromodulation precision and flexibility.

Advancing our understanding of brain function and developing treatments for neurological diseases hinge on the ability to modulate neuronal groups in specific brain areas without invasive techniques. Here, we introduce Airy-beam holographic sonogenetics (AhSonogenetics) as an implant-free, cell type-specific, spatially precise, and flexible neuromodulation approach in freely moving mice. AhSonogenetics utilizes wearable ultrasound devices manufactured using 3D-printed Airy-beam holographic metasurfaces. These devices are designed to manipulate neurons genetically engineered to express ultrasound-sensitive ion channels, enabling precise modulation of specific neuronal populations. By dynamically steering the focus of Airy beams through ultrasound frequency tuning, AhSonogenetics is capable of modulating neuronal populations within specific subregions of the striatum. One notable feature of AhSonogenetics is its ability to flexibly stimulate either the left or right striatum in a single mouse. This flexibility is achieved by simply switching the acoustic metasurface in the wearable ultrasound device, eliminating the need for multiple implants or interventions. AhSonogentocs also integrates seamlessly with in vivo calcium recording via fiber photometry, showcasing its compatibility with optical modalities without cross talk. Moreover, AhSonogenetics can generate double foci for bilateral stimulation and alleviate motor deficits in Parkinson's disease mice. This advancement is significant since many neurological disorders, including Parkinson's disease, involve dysfunction in multiple brain regions. By enabling precise and flexible cell type-specific neuromodulation without invasive procedures, AhSonogenetics provides a powerful tool for investigating intact neural circuits and offers promising interventions for neurological disorders.

Detecting material state changes in the nucleolus by label-free digital holographic microscopy.

Ribosome biogenesis is initiated in the nucleolus, a multiphase biomolecular condensate formed by liquid-liquid phase separation. The nucleolus is a powerful disease biomarker and stress biosensor whose morphology reflects function. Here we have used digital holographic microscopy (DHM), a label-free quantitative phase contrast microscopy technique, to detect nucleoli in adherent and suspension human cells. We trained convolutional neural networks to detect and quantify nucleoli automatically on DHM images. Holograms containing cell optical thickness information allowed us to define a novel index which we used to distinguish nucleoli whose material state had been modulated optogenetically by blue-light-induced protein aggregation. Nucleoli whose function had been impacted by drug treatment or depletion of ribosomal proteins could also be distinguished. We explored the potential of the technology to detect other natural and pathological condensates, such as those formed upon overexpression of a mutant form of huntingtin, ataxin-3, or TDP-43, and also other cell assemblies (lipid droplets). We conclude that DHM is a powerful tool for quantitatively characterizing nucleoli and other cell assemblies, including their material state, without any staining.

Fast optical recording of neuronal activity by three-dimensional custom-access serial holography.

Optical recording of neuronal activity in three-dimensional (3D) brain circuits at cellular and millisecond resolution in vivo is essential for probing information flow in the brain. While random-access multiphoton microscopy permits fast optical access to neuronal targets in three dimensions, the method is challenged by motion artifacts when recording from behaving animals. Therefore, we developed three-dimensional custom-access serial holography (3D-CASH). Built on a fast acousto-optic light modulator, 3D-CASH performs serial sampling at 40 kHz from neurons at freely selectable 3D locations. Motion artifacts are eliminated by targeting each neuron with a size-optimized pattern of excitation light covering the cell body and its anticipated displacement field. Spike rates inferred from GCaMP6f recordings in visual cortex of awake mice tracked the phase of a moving bar stimulus with higher spike correlation between intra compared to interlaminar neuron pairs. 3D-CASH offers access to the millisecond correlation structure of in vivo neuronal activity in 3D microcircuits.

Label-Free Single Nanoparticle Identification and Characterization in Demanding Environment, Including Infectious Emergent Virus.

Unknown particle screening-including virus and nanoparticles-are keys in medicine, industry, and also in water pollutant determination. Here, RYtov MIcroscopy for Nanoparticles Identification (RYMINI) is introduced, a staining-free, non-invasive, and non-destructive optical approach that is merging holographic label-free 3D tracking with high-sensitivity quantitative phase imaging into a compact optical setup. Dedicated to the identification and then characterization of single nano-object in solution, it is compatible with highly demanding environments, such as level 3 biological laboratories, with high resilience to external source of mechanical and optical noise. Metrological characterization is performed at the level of each single particle on both absorbing and transparent particles as well as on immature and infectious HIV, SARS-CoV-2 and extracellular vesicles in solution. The capability of RYMINI to determine the nature, concentration, size, complex refractive index and mass of each single particle without knowledge or model of the particles' response is demonstrated. The system surpasses 90% accuracy for automatic identification between dielectric/metallic/biological nanoparticles and ≈80% for intraclass chemical determination of metallic and dielectric. It falls down to 50-70% for type determination inside the biological nanoparticle's class.

Label-free, high-throughput holographic imaging to evaluate mammalian gametes and embryos†.

Assisted reproduction is one of the significant tools to treat human infertility. Morphological assessment is the primary method to determine sperm and embryo viability during in vitro fertilization cycles. It has the advantage of being a quick, convenient, and inexpensive means of assessment. However, visual observation is of limited predictive value for early embryo morphology. It has led many to search for other imaging tools to assess the reproductive potential of a given embryo. The limitations of visual assessment apply to both humans and animals. One recent innovation in assisted reproduction technology imaging is interferometric phase microscopy, also known as holographic microscopy. Interferometric phase microscopy/quantitative phase imaging is the next likely progression of analytical microscopes for the assisted reproduction laboratory. The interferometric phase microscopy system analyzes waves produced by the light as it passes through the specimen observed. The microscope collects the light waves produced and uses the algorithm to create a hologram of the specimen. Recently, interferometric phase microscopy has been combined with quantitative phase imaging, which joins phase contrast microscopy with holographic microscopy. These microscopes collect light waves produced and use the algorithm to create a hologram of the specimen. Unlike other systems, interferometric phase microscopy can provide a quantitative digital image, and it can make 2D and 3D images of the samples. This review summarizes some newer and more promising quantitative phase imaging microscopy systems for evaluating gametes and embryos. Studies clearly show that quantitative phase imaging is superior to bright field microscopy-based evaluation methods when evaluating sperm and oocytes prior to IVF and embryos prior to transfer. However, further assessment of these systems for efficacy, reproducibility, cost-effectiveness, and embryo/gamete safety must take place before they are widely adopted.

Comparison of holotomographic microscopy and coherence-controlled holographic microscopy.

Quantitative phase imaging (QPI) is a powerful tool for label-free visualisation of living cells. Here, we compare two QPI microscopes - the Telight Q-Phase microscope and the Nanolive 3D Cell Explorer-fluo microscope. Both systems provide unbiased information about cell morphology, such as individual cell dry mass, perimeter and area. The Q-Phase microscope uses artefact-free, coherence-controlled holographic imaging technology to visualise cells in real time with minimal phototoxicity. The 3D Cell Explorer-fluo employs laser-based holotomography to reconstruct 3D images of living cells, visualising their internal structures and dynamics. Here, we analysed the strengths and limitations of both microscopes when examining two morphologically distinct cell lines - the cuboidal epithelial MDCK cells which form multicellular clusters and solitary growing Rat2 fibroblasts. We focus mainly on the ability of the devices to generate images suitable for single-cell segmentation by the built-in software, and we discuss the segmentation results and quantitative data generated from the segmented images. We show that both microscopes offer slightly different advantages, and the choice between them depends on the specific requirements and goals of the user.

Low-energy electron holography imaging of conformational variability of single-antibody molecules from electrospray ion beam deposition.

Imaging of proteins at the single-molecule level can reveal conformational variability, which is essential for the understanding of biomolecules. To this end, a biologically relevant state of the sample must be retained during both sample preparation and imaging. Native electrospray ionization (ESI) can transfer even the largest protein complexes into the gas phase while preserving their stoichiometry and overall shape. High-resolution imaging of protein structures following native ESI is thus of fundamental interest for establishing the relation between gas phase and solution structure. Taking advantage of low-energy electron holography's (LEEH) unique capability of imaging individual proteins with subnanometer resolution, we investigate the conformational flexibility of Herceptin, a monoclonal IgG antibody, deposited by native electrospray mass-selected ion beam deposition (ES-IBD) on graphene. Images reconstructed from holograms reveal a large variety of conformers. Some of these conformations can be mapped to the crystallographic structure of IgG, while others suggest that a compact, gas-phase-related conformation, adopted by the molecules during ES-IBD, is retained. We can steer the ratio of those two types of conformations by changing the landing energy of the protein on the single-layer graphene surface. Overall, we show that LEEH can elucidate the conformational heterogeneity of inherently flexible proteins, exemplified here by IgG antibodies, and thereby distinguish gas-phase collapse from rearrangement on surfaces.

Effect of sound-induced vibrations of the pinna on head-related transfer functions: Experimental and numerical investigations.

Numerical simulations of head-related transfer functions (HRTFs) conventionally assume a rigid boundary condition for the pinna. The human pinna, however, is an elastic deformable body that can vibrate due to incident acoustic waves. This work investigates how sound-induced vibrations of the pinna can affect simulated HRTF magnitudes. The work will motivate the research question by measuring the sound-induced vibrational patterns of an artificial pinna with a high-speed holographic interferometric system. Then, finite element simulations are used to determine HRTFs for a tabletop model of the B&K 5128 head and torso simulator for a number of directions. Two scenarios are explored: one where the pinna is modeled as perfectly rigid, and another where the pinna is modeled as linear elastic with material properties close to that of auricular cartilage. The findings suggest that pinna vibrations have negligible effects on HRTF magnitudes up to 5 kHz. The same conclusion, albeit with less certainty, is drawn for higher frequencies. Finally, the importance of the elastic domain's material properties is emphasized and possible implications for validation studies on dummy heads 1as well as the limitations of the present work are discussed in detail.

Delivering clinical tutorials to medical students using the Microsoft HoloLens 2: A mixed-methods evaluation.

BACKGROUND: Mixed reality offers potential educational advantages in the delivery of clinical teaching. Holographic artefacts can be rendered within a shared learning environment using devices such as the Microsoft HoloLens 2. In addition to facilitating remote access to clinical events, mixed reality may provide a means of sharing mental models, including the vertical and horizontal integration of curricular elements at the bedside. This study aimed to evaluate the feasibility of delivering clinical tutorials using the Microsoft HoloLens 2 and the learning efficacy achieved. METHODS: Following receipt of institutional ethical approval, tutorials on preoperative anaesthetic history taking and upper airway examination were facilitated by a tutor who wore the HoloLens device. The tutor interacted face to face with a patient and two-way audio-visual interaction was facilitated using the HoloLens 2 and Microsoft Teams with groups of students who were located in a separate tutorial room. Holographic functions were employed by the tutor. The tutor completed the System Usability Scale, the tutor, technical facilitator, patients, and students provided quantitative and qualitative feedback, and three students participated in semi-structured feedback interviews. Students completed pre- and post-tutorial, and end-of-year examinations on the tutorial topics. RESULTS: Twelve patients and 78 students participated across 12 separate tutorials. Five students did not complete the examinations and were excluded from efficacy calculations. Student feedback contained 90 positive comments, including the technology's ability to broadcast the tutor's point-of-vision, and 62 negative comments, where students noted issues with the audio-visual quality, and concerns that the tutorial was not as beneficial as traditional in-person clinical tutorials. The technology and tutorial structure were viewed favourably by the tutor, facilitator and patients. Significant improvement was observed between students' pre- and post-tutorial MCQ scores (mean 59.2% Vs 84.7%, p < 0.001). CONCLUSIONS: This study demonstrates the feasibility of using the HoloLens 2 to facilitate remote bedside tutorials which incorporate holographic learning artefacts. Students' examination performance supports substantial learning of the tutorial topics. The tutorial structure was agreeable to students, patients and tutor. Our results support the feasibility of offering effective clinical teaching and learning opportunities using the HoloLens 2. However, the technical limitations and costs of the device are significant, and further research is required to assess the effectiveness of this tutorial format against in-person tutorials before wider roll out of this technology can be recommended as a result of this study.

Live Cell Imaging by Single-Shot Common-Path Wide Field-of-View Reflective Digital Holographic Microscope.

Quantitative phase imaging by digital holographic microscopy (DHM) is a nondestructive and label-free technique that has been playing an indispensable role in the fields of science, technology, and biomedical imaging. The technique is competent in imaging and analyzing label-free living cells and investigating reflective surfaces. Herein, we introduce a new configuration of a wide field-of-view single-shot common-path off-axis reflective DHM for the quantitative phase imaging of biological cells that leverages several advantages, including being less-vibration sensitive to external perturbations due to its common-path configuration, also being compact in size, simple in optical design, highly stable, and cost-effective. A detailed description of the proposed DHM system, including its optical design, working principle, and capability for phase imaging, is presented. The applications of the proposed system are demonstrated through quantitative phase imaging results obtained from the reflective surface (USAF resolution test target) as well as transparent samples (living plant cells). The proposed system could find its applications in the investigation of several biological specimens and the optical metrology of micro-surfaces.

Label-Free Three-Dimensional Morphological Characterization of Cell Death Using Holographic Tomography.

This study presents a novel label-free approach for characterizing cell death states, eliminating the need for complex molecular labeling that may yield artificial or ambiguous results due to technical limitations in microscope resolution. The proposed holographic tomography technique offers a label-free avenue for capturing precise three-dimensional (3D) refractive index morphologies of cells and directly analyzing cellular parameters like area, height, volume, and nucleus/cytoplasm ratio within the 3D cellular model. We showcase holographic tomography results illustrating various cell death types and elucidate distinctive refractive index correlations with specific cell morphologies complemented by biochemical assays to verify cell death states. These findings hold promise for advancing in situ single cell state identification and diagnosis applications.

Holographic Brain Theory: Super-Radiance, Memory Capacity and Control Theory.

We investigate Quantum Electrodynamics corresponding to the holographic brain theory introduced by Pribram to describe memory in the human brain. First, we derive a super-radiance solution in Quantum Electrodynamics with non-relativistic charged bosons (a model of molecular conformational states of water) for coherent light sources of holograms. Next, we estimate memory capacity of a brain neocortex, and adopt binary holograms to manipulate optical information. Finally, we introduce a control theory to manipulate holograms involving biological water's molecular conformational states. We show how a desired waveform in holography is achieved in a hierarchical model using numerical simulations.

[Simulation and Navigation in the Neurosurgical Field Using Three-Dimensional Hholograms by Mixed Reality Devices].

Recently, three-dimensional(3D)holograms from mixed-reality(MR)devices have become available in the medical field. 3D holographic images can provide immersive and intuitive information that has been reported to be very useful for preoperative simulations. Compared with conventional 3D images on a two-dimensional(2D)monitor, 3D holograms offer a higher level of realism, allowing observation of the images anytime and anywhere if the MR device is operational. Even during surgery, surgeons can check realistic 3D holograms in front of them, above the surgical field, without having to turn their heads toward a 2D monitor on the wall. 3D holograms can also be used for neuronavigation if the hologram is tracked to the patient's real head. This method can be defined as 3D augmented reality(AR)navigation, which shows a hologram of a target, such as a tumor or aneurysm, inside the head and brain. In the future, interventions using these techniques with 3D holograms from MR devices are expected to evolve and develop new types of treatments for endoscopic surgery or fluoroscopy-guided endovascular surgery.

Development of serial X-ray fluorescence holography for radiation-sensitive protein crystals.

X-ray fluorescence holography (XFH) is a powerful atomic resolution technique capable of directly imaging the local atomic structure around atoms of a target element within a material. Although it is theoretically possible to use XFH to study the local structures of metal clusters in large protein crystals, the experiment has proven difficult to perform, especially on radiation-sensitive proteins. Here, the development of serial X-ray fluorescence holography to allow the direct recording of hologram patterns before the onset of radiation damage is reported. By combining a 2D hybrid detector and the serial data collection used in serial protein crystallography, the X-ray fluorescence hologram can be directly recorded in a fraction of the measurement time needed for conventional XFH measurements. This approach was demonstrated by obtaining the Mn Kα hologram pattern from the protein crystal Photosystem II without any X-ray-induced reduction of the Mn clusters. Furthermore, a method to interpret the fluorescence patterns as real-space projections of the atoms surrounding the Mn emitters has been developed, where the surrounding atoms produce large dark dips along the emitter-scatterer bond directions. This new technique paves the way for future experiments on protein crystals that aim to clarify the local atomic structures of their functional metal clusters, and for other related XFH experiments such as valence-selective XFH or time-resolved XFH.

Micropipette aspiration as a tool for single-particle X-ray imaging and diffraction.

A sample environment and manipulation tool is presented for single-particle X-ray experiments in an aqueous environment. The system is based on a single water droplet, positioned on a substrate that is structured by a hydrophobic and hydrophilic pattern to stabilize the droplet position. The substrate can support several droplets at a time. Evaporation is prevented by covering the droplet by a thin film of mineral oil. In this windowless fluid which minimizes background signal, single particles can be probed and manipulated by micropipettes, which can easily be inserted and steered in the droplet. Holographic X-ray imaging is shown to be well suited to observe and monitor the pipettes, as well as the droplet surface and the particles. Aspiration and force generation are also enabled based on an application of controlled pressure differences. Experimental challenges are addressed and first results are presented, obtained at two different undulator endstations with nano-focused beams. Finally, the sample environment is discussed in view of future coherent imaging and diffraction experiments with synchrotron radiation and single X-ray free-electron laser pulses.

On monocytes and lymphocytes biolens clustering by in flow holographic microscopy.

Live cells act as biological lenses and can be employed as real-world optical components in bio-hybrid systems. Imaging at nanoscale, optical tweezers, lithography and also photonic waveguiding are some of the already proven functionalities, boosted by the advantage that cells are fully biocompatible for intra-body applications. So far, various cell types have been studied for this purpose, such as red blood cells, bacterial cells, stem cells and yeast cells. White Blood Cells (WBCs) play a very important role in the regulation of the human body activities and are usually monitored for assessing its health. WBCs can be considered bio-lenses but, to the best of our knowledge, characterization of their optical properties have not been investigated yet. Here, we report for the first time an accurate study of two model classes of WBCs (i.e., monocytes and lymphocytes) by means of a digital holographic microscope coupled with a microfluidic system, assuming WBCs bio-lens characteristics. Thus, quantitative phase maps for many WBCs have been retrieved in flow-cytometry (FC) by achieving a significant statistical analysis to prove the enhancement in differentiation among sphere-like bio-lenses according to their sizes (i.e., diameter d) exploiting intensity parameters of the modulated light in proximity of the cell optical axis. We show that the measure of the low intensity area (S: I z < I th z ) in a fixed plane, is a feasible parameter for cell clustering, while achieving robustness against experimental misalignments and allowing to adjust the measurement sensitivity in post-processing. 2D scatterplots of the identified parameters (d-S) show better differentiation respect to the 1D case. The results show that the optical focusing properties of WBCs allow the clustering of the two populations by means of a mere morphological analysis, thus leading to the new concept of cell-optical-fingerprint avoiding fluorescent dyes. This perspective can open new routes in biomedical sciences, such as the chance to find optical-biomarkers at single cell level for label-free diagnosis.

Fast 2-photon stimulation using holographic patterns.

Two decades after its introduction, optogenetics - a biological technique to control the activity of neurons or other cell types with light - remains a cutting edge and promising tool to study biological processes. Its increasing usage in research varies widely from causally exploring biological mechanisms and neural computations, to neurostimulation and sensory restauration. To stimulate neurons in the brain, a variety of approaches have been developed to generate precise spatiotemporal light patterns. Yet certain constrains still exists in the current optical techniques to activate a neuronal population with both cellular resolution and millisecond precision. Here, we describe an experimental setup allowing to stimulate a few tens of neurons in a plane at sub-millisecond rates using 2-photon activation. A liquid crystal on silicon spatial light modulator (LCoS-SLM) was used to generate spatial patterns in 2 dimensions. The image of the patterns was formed on the plane of a digital micromirror device (DMD) that was used as a fast temporal modulator of each region of interest. Using fluorescent microscopy and patch-clamp recording of neurons in culture expressing the light-gated ion channels, we characterized the temporal and spatial resolution of the microscope. We described the advantages of combining the LCoS-SLM with the DMD to maximize the temporal precision, modulate the illumination amplitude, and reduce background activation. Finally, we showed that this approach can be extended to patterns in 3 dimensions. We concluded that the methodology is well suited to address important questions about the role of temporal information in neuronal coding.