Towards real-time photorealistic 3D holography with deep neural networks.

The ability to present three-dimensional (3D) scenes with continuous depth sensation has a profound impact on virtual and augmented reality, human-computer interaction, education and training. Computer-generated holography (CGH) enables high-spatio-angular-resolution 3D projection via numerical simulation of diffraction and interference1. Yet, existing physically based methods fail to produce holograms with both per-pixel focal control and accurate occlusion2,3. The computationally taxing Fresnel diffraction simulation further places an explicit trade-off between image quality and runtime, making dynamic holography impractical4. Here we demonstrate a deep-learning-based CGH pipeline capable of synthesizing a photorealistic colour 3D hologram from a single RGB-depth image in real time. Our convolutional neural network (CNN) is extremely memory efficient (below 620 kilobytes) and runs at 60 hertz for a resolution of 1,920 × 1,080 pixels on a single consumer-grade graphics processing unit. Leveraging low-power on-device artificial intelligence acceleration chips, our CNN also runs interactively on mobile (iPhone 11 Pro at 1.1 hertz) and edge (Google Edge TPU at 2.0 hertz) devices, promising real-time performance in future-generation virtual and augmented-reality mobile headsets. We enable this pipeline by introducing a large-scale CGH dataset (MIT-CGH-4K) with 4,000 pairs of RGB-depth images and corresponding 3D holograms. Our CNN is trained with differentiable wave-based loss functions5 and physically approximates Fresnel diffraction. With an anti-aliasing phase-only encoding method, we experimentally demonstrate speckle-free, natural-looking, high-resolution 3D holograms. Our learning-based approach and the Fresnel hologram dataset will help to unlock the full potential of holography and enable applications in metasurface design6,7, optical and acoustic tweezer-based microscopic manipulation8-10, holographic microscopy11 and single-exposure volumetric 3D printing12,13.

Local augmented graph neural network for multi-omics cancer prognosis prediction and analysis.

Cancer prognosis prediction and analysis can help patients understand expected life and help clinicians provide correct therapeutic guidance. Thanks to the development of sequencing technology, multi-omics data, and biological networks have been used for cancer prognosis prediction. Besides, graph neural networks can simultaneously consider multi-omics features and molecular interactions in biological networks, becoming mainstream in cancer prognosis prediction and analysis. However, the limited number of neighboring genes in biological networks restricts the accuracy of graph neural networks. To solve this problem, a local augmented graph convolutional network named LAGProg is proposed in this paper for cancer prognosis prediction and analysis. The process follows: first, given a patient's multi-omics data features and biological network, the corresponding augmented conditional variational autoencoder generates features. Then, the generated augmented features and the original features are fed into a cancer prognosis prediction model to complete the cancer prognosis prediction task. The conditional variational autoencoder consists of two parts: encoder-decoder. In the encoding phase, an encoder learns the conditional distribution of the multi-omics data. As a generative model, a decoder takes the conditional distribution and the original feature as inputs to generate the enhanced features. The cancer prognosis prediction model consists of a two-layer graph convolutional neural network and a Cox proportional risk network. The Cox proportional risk network consists of fully connected layers. Extensive experiments on 15 real-world datasets from TCGA demonstrated the effectiveness and efficiency of the proposed method in predicting cancer prognosis. LAGProg improved the C-index values by an average of 8.5% over the state-of-the-art graph neural network method. Moreover, we confirmed that the local augmentation technique could enhance the model's ability to represent multi-omics features, improve the model's robustness to missing multi-omics features, and prevent the model's over-smoothing during training. Finally, based on genes identified through differential expression analysis, we discovered 13 prognostic markers highly associated with breast cancer, among which ten genes have been proved by literature review.

Physical Realization of Elastic Cloaking with a Polar Material.

An elastic cloak is a coating material that can be applied to an arbitrary inclusion to make it indistinguishable from the background medium. Cloaking against elastic disturbances, in particular, has been demonstrated using several designs and gauges. None, however, tolerate the coexistence of normal and shear stresses due to a shortage of physical realization of transformation-invariant elastic materials. Here, we overcome this limitation to design and fabricate a new class of polar materials with a distribution of body torque that exhibits asymmetric stresses. A static cloak for full two-dimensional elasticity is thus constructed based on the transformation method. The proposed cloak is made of a functionally graded multilayered lattice embedded in an isotropic continuum background. While one layer is tailored to produce a target elastic behavior, the other layers impose a set of kinematic constraints equivalent to a distribution of body torque that breaks the stress symmetry. Experimental testing under static compressive and shear loads demonstrates encouraging cloaking performance in good agreement with our theoretical prediction. The work sets a precedent in the field of transformation elasticity and should find applications in mechanical stress shielding and stealth technologies.

Nonlinear Parameter Identification of a Resonant Electrostatic MEMS Actuator.

We experimentally investigate the primary superharmonic of order two and subharmonic of order one-half resonances of an electrostatic MEMS actuator under direct excitation. We identify the parameters of a one degree of freedom (1-DOF) generalized Duffing oscillator model representing it. The experiments were conducted in soft vacuum to reduce squeeze-film damping, and the actuator response was measured optically using a laser vibrometer. The predictions of the identified model were found to be in close agreement with the experimental results. We also identified the noise spectral density of process (actuation voltage) and measurement noise.

Computational discovery of microstructured composites with optimal stiffness-toughness trade-offs.

The conflict between stiffness and toughness is a fundamental problem in engineering materials design. However, the systematic discovery of microstructured composites with optimal stiffness-toughness trade-offs has never been demonstrated, hindered by the discrepancies between simulation and reality and the lack of data-efficient exploration of the entire Pareto front. We introduce a generalizable pipeline that integrates physical experiments, numerical simulations, and artificial neural networks to address both challenges. Without any prescribed expert knowledge of material design, our approach implements a nested-loop proposal-validation workflow to bridge the simulation-to-reality gap and find microstructured composites that are stiff and tough with high sample efficiency. Further analysis of Pareto-optimal designs allows us to automatically identify existing toughness enhancement mechanisms, which were previously found through trial and error or biomimicry. On a broader scale, our method provides a blueprint for computational design in various research areas beyond solid mechanics, such as polymer chemistry, fluid dynamics, meteorology, and robotics.

LncRNA RAD51-AS1 Regulates Human Bone Marrow Mesenchymal Stem Cells via Interaction with YBX1 to Ameliorate Osteoporosis.

Long noncoding RNA (lncRNA) is a new key regulatory molecule in the occurrence of osteoporosis, but its research is still in the primary stage. In order to study the role and mechanism of lncRNA in the occurrence of osteoporosis, we reannotated the GSE35956 datasets, compared and analyzed the differential expression profiles of lncRNAs between bone marrow mesenchymal stem cells (hBMSCs) from healthy and osteoporotic patients, and then screened a lncRNA RAD51-AS1 with low expression in hBMSCs from osteoporotic patients, and its role in the occurrence of osteoporosis has not been studied. We confirmed that the expression level of lncRNA RAD51-AS1 in hBMSCs from patients with osteoporosis was significantly lower than those from healthy donors. A nuclear cytoplasmic separation experiment and RNA fluorescence in situ hybridization showed that RAD51-AS1 was mainly located in the nucleus. RAD51-AS1 knockdown significantly inhibited the proliferation and osteogenic differentiation of hBMSCs and significantly increased their apoptosis, while RAD51-AS1 overexpression significantly promoted the proliferation, osteogenic differentiation, and ectopic bone formation of hBMSCs. Mechanistically, we found that RAD51-AS1 banded to YBX1 and then activated the TGF-ß signal pathway by binding to Smad7 and Smurf2 mRNA to inhibit their translation and transcription up-regulated PCNA and SIVA1 by binding to their promoter regions. In conclusion, RAD51-AS1 promoted the proliferation and osteogenic differentiation of hBMSCs by binding YBX1, inhibiting the translation of Smad7 and Smurf2, and transcriptionally up-regulated PCNA and SIVA1.

End-to-end learning of 3D phase-only holograms for holographic display.

Computer-generated holography (CGH) provides volumetric control of coherent wavefront and is fundamental to applications such as volumetric 3D displays, lithography, neural photostimulation, and optical/acoustic trapping. Recently, deep learning-based methods emerged as promising computational paradigms for CGH synthesis that overcome the quality-runtime tradeoff in conventional simulation/optimization-based methods. Yet, the quality of the predicted hologram is intrinsically bounded by the dataset's quality. Here we introduce a new hologram dataset, MIT-CGH-4K-V2, that uses a layered depth image as a data-efficient volumetric 3D input and a two-stage supervised+unsupervised training protocol for direct synthesis of high-quality 3D phase-only holograms. The proposed system also corrects vision aberration, allowing customization for end-users. We experimentally show photorealistic 3D holographic projections and discuss relevant spatial light modulator calibration procedures. Our method runs in real-time on a consumer GPU and 5 FPS on an iPhone 13 Pro, promising drastically enhanced performance for the applications above.

Multiscale Temporal Self-Attention and Dynamical Graph Convolution Hybrid Network for EEG-Based Stereogram Recognition.

Stereopsis is the ability of human beings to get the 3D perception on real scenarios. The conventional stereopsis measurement is based on subjective judgment for stereograms, leading to be easily affected by personal consciousness. To alleviate the issue, in this paper, the EEG signals evoked by dynamic random dot stereograms (DRDS) are collected for stereogram recognition, which can help the ophthalmologists diagnose strabismus patients even without real-time communication. To classify the collected Electroencephalography (EEG) signals, a novel multi-scale temporal self-attention and dynamical graph convolution hybrid network (MTS-DGCHN) is proposed, including multi-scale temporal self-attention module, dynamical graph convolution module and classification module. Firstly, the multi-scale temporal self-attention module is employed to learn time continuity information, where the temporal self-attention block is designed to highlight the global importance of each time segments in one EEG trial, and the multi-scale convolution block is developed to further extract advanced temporal features in multiple receptive fields. Meanwhile, the dynamical graph convolution module is utilized to capture spatial functional relationships between different EEG electrodes, in which the adjacency matrix of each GCN layer is adaptively tuned to explore the optimal intrinsic relationship. Finally, the temporal and spatial features are fed into the classification module to obtain prediction results. Extensive experiments are conducted on collected datasets i.e., SRDA and SRDB, and the results demonstrate the proposed MTS-DGCHN achieves outstanding classification performance compared with the other methods. The datasets are available at https://github.com/YANGeeg/TJU-SRD-datasets and the code is at https://github.com/YANGeeg/MTS-DGCHN.

Extracellular vesicles derived from LPS-preconditioned human synovial mesenchymal stem cells inhibit extracellular matrix degradation and prevent osteoarthritis of the knee in a mouse model.

BACKGROUND: Previous studies report that lipopolysaccharide (LPS)-preconditioned mesenchymal stem cells have enhanced trophic support and improved regenerative and repair properties. Extracellular vesicles secreted by synovial mesenchymal stem cells (EVs) can reduce cartilage damage caused by osteoarthritis (OA). Previous studies show that extracellular vesicles secreted by LPS-preconditioned synovial mesenchymal stem cells (LPS-pre EVs) can improve the response to treatment of osteoarthritis (OA). This study sought to explore effects of LPS-pre EVs on chondrocyte proliferation, migration, and chondrocyte apoptosis, as well as the protective effect of LPS-pre EVs on mouse articular cartilage. METHODS: Chondrocytes were extracted to explore the effect of LPS-pre EVs on proliferation, migration, and apoptosis of chondrocytes. In addition, the effect of LPS-pre EVs on expression level of important proteins of chondrocytes was explored suing in vitro experiments. Further, intraarticular injection of LPS-pre EVs was performed on the destabilization of the medial meniscus (DMM)-induced mouse models of OA to explore the therapeutic effect of LPS-pre EVs on osteoarthritis in vivo. RESULTS: Analysis showed that LPS-pre EVs significantly promoted proliferation and migration of chondrocytes and inhibited the apoptosis of chondrocytes compared with PBS and EVs. Moreover, LPS-pre EVs inhibited decrease of aggrecan and COL2A1 and increase of ADAMTS5 caused by IL-1ß through let-7b. Furthermore, LPS-pre EVs significantly prevented development of OA in DMM-induced mouse models of OA. CONCLUSIONS: LPS pretreatment is an effective and promising method to improve therapeutic effect of extracellular vesicles secreted from SMSCs on OA.

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors.

We present a novel approach to integrate data from multiple sensor types for dense 3D reconstruction of indoor scenes in realtime. Existing algorithms are mainly based on a single RGBD camera and thus require continuous scanning of areas with sufficient geometric features. Otherwise, tracking may fail due to unreliable frame registration. Inspired by the fact that the fusion of multiple sensors can combine their strengths towards a more robust and accurate self-localization, we incorporate multiple types of sensors which are prevalent in modern robot systems, including a 2D range sensor, an inertial measurement unit (IMU), and wheel encoders. We fuse their measurements to reinforce the tracking process and to eventually obtain better 3D reconstructions. Specifically, we develop a 2D truncated signed distance field (TSDF) volume representation for the integration and ray-casting of laser frames, leading to a unified cost function in the pose estimation stage. For validation of the estimated poses in the loop-closure optimization process, we train a classifier for the features extracted from heterogeneous sensors during the registration progress. To evaluate our method on challenging use case scenarios, we assembled a scanning platform prototype to acquire real-world scans. We further simulated synthetic scans based on high-fidelity synthetic scenes for quantitative evaluation. Extensive experimental evaluation on these two types of scans demonstrate that our system is capable of robustly acquiring dense 3D reconstructions and outperforms state-of-the-art RGBD and LiDAR systems.