NeRFBuff: Fast Neural Rendering via Inter-frame Feature Buffering.

Neural radiance fields (NeRF) have demonstrated impressive performance in novel view synthesis, but are still slow to render complex scenes at a high resolution. We introduce a novel method to boost the NeRF rendering speed by utilizing the temporal coherence between consecutive frames. Rather than computing features of each frame entirely from scratch, we reuse the coherent information (e.g., density and color) computed from the previous frames to help render the current frame, which significantly boosts rendering speed. To effectively manage the coherent information of previous frames, we introduce a history buffer with a multiple-plane structure, which is built online and updated from old frames to new frames. We name this buffer as multiple plane buffer (MPB). With this MPB, a new frame can be efficiently rendered using the warped features from previous frames. Extensive experiments on the NeRF-Synthetic, LLFF, and Mip-NeRF-360 datasets demonstrate that our method significantly boosts rendering efficiency and achieves 4× speedup on real-world scenes compared to the baseline methods while preserving competitive rendering quality.

Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation.

We introduce Metric3D v2, a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image, which is crucial for metric 3D recovery. While depth and normal are geometrically related and highly complimentary, they present distinct challenges. State-of-the-art (SoTA) monocular depth methods achieve zero-shot generalization by learning affine-invariant depths, which cannot recover real-world metrics. Meanwhile, SoTA normal estimation methods have limited zero-shot performance due to the lack of large-scale labeled data. To tackle these issues, we propose solutions for both metric depth estimation and surface normal estimation. For metric depth estimation, we show that the key to a zero-shot single-view model lies in resolving the metric ambiguity from various camera models and large-scale data training. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problem and can be effortlessly plugged into existing monocular models. For surface normal estimation, we propose a joint depth-normal optimization module to distill diverse data knowledge from metric depth, enabling normal estimators to learn beyond normal labels. Equipped with these modules, our depth-normal models can be stably trained with over 16 million of images from thousands of camera models with different-type annotations, resulting in zero-shot generalization to in-the-wild images with unseen camera settings. Our method currently ranks the 1st on various zero-shot and non-zero-shot benchmarks for metric depth, affine-invariant-depth as well as surface-normal prediction, shown in Fig. 1. Notably, we surpassed the ultra-recent MarigoldDepth and DepthAnything on various depth benchmarks including NYUv2 and KITTI. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. The potential benefits extend to downstream tasks, which can be significantly improved by simply plugging in our model. For example, our model relieves the scale drift issues of monocular-SLAM (Fig. 3), leading to high-quality metric scale dense mapping. These applications highlight the versatility of Metric3D v2 models as geometric foundation models. Our project page is at https://JUGGHM.github.io/Metric3Dv2.

Adaptive Surface Normal Constraint for Geometric Estimation From Monocular Images.

We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context. The difficulty of reliably capturing geometric context in existing methods impedes their ability to accurately enforce the consistency between the different geometric properties, thereby leading to a bottleneck of geometric estimation quality. We therefore propose the Adaptive Surface Normal (ASN) constraint, a simple yet efficient method. Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints. By dynamically determining reliable local geometry from randomly sampled candidates, we establish a surface normal constraint, where the validity of these candidates is evaluated using the geometric context. Furthermore, our normal estimation leverages the geometric context to prioritize regions that exhibit significant geometric variations, which makes the predicted normals accurately capture intricate and detailed geometric information. Through the integration of geometric context, our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images. We validate the superiority of our approach over state-of-the-art methods through extensive evaluations and comparisons on diverse indoor and outdoor datasets, showcasing its efficiency and robustness.

Neural cell differentiation in vitro from adult human bone marrow mesenchymal stem cells.

Bone marrow (BM) mesenchymal stem cells (MSCs) are cells capable of expanding and differentiating in vitro into nonhematopoietic cells. Neurotrophic cytokines, such as human epidermal growth factor (hEGF) and bovine fibroblast growth factor (bFGF) can induce differentiation into neural cells (NCs). When BM MSCs were cultured with hEGF and bFGF, RNA expression of neuronal specific markers Nestin, MAP-2, and tyrosine hydroxylase (TH) were observed. We tested a new cytokine combination to generate mature NCs. The plastic-adherent cells were collected and then split when they were 90% confluent from an enriched mononuclear cell layer. At passage 3, MSCs were cultured in neural differentiation media (dbcAMP, IBMX, FGF-8, BDNF, hEGF, and bFGF in NEUROBASAL media plus B27). Cells were counted on day 6. Immunofluorescent staining and reverse transcriptase (RT)-PCR were performed to evaluate the expression of neural markers. On day 6, 66% of cells developed dendrites and presented typical neural cell morphology. Some cells were positive for early neural markers Nestin and beta-tubulin III. Cells expressing mature neuronal markers (NF, NeuN, Tau, Nurr1, GABA, oligodendryte GalC, and glial GFAP) were also seen. By adding hEGF, bFGF, dbcAMP, IBMX, BDNF, and bFGF-8 into NEUROBASAL media plus B27, BM MSCs were directed toward becoming early and mature NCs.