Surface Reconstruction from Structured Light Images Using Differentiable Rendering.

When 3D scanning objects, the objective is usually to obtain a continuous surface. However, most surface scanning methods, such as structured light scanning, yield a point cloud. Obtaining a continuous surface from a point cloud requires a subsequent surface reconstruction step, which is directly affected by any error from the computation of the point cloud. In this work, we propose a one-step approach in which we compute the surface directly from structured light images. Our method minimizes the least-squares error between photographs and renderings of a triangle mesh, where the vertex positions of the mesh are the parameters of the minimization problem. To ensure fast iterations during optimization, we use differentiable rendering, which computes images and gradients in a single pass. We present simulation experiments demonstrating that our method for computing a triangle mesh has several advantages over approaches that rely on an intermediate point cloud. Our method can produce accurate reconstructions when initializing the optimization from a sphere. We also show that our method is good at reconstructing sharp edges and that it is robust with respect to image noise. In addition, our method can improve the output from other reconstruction algorithms if we use these for initialization.

Alignment of rendered images with photographs for testing appearance models.

We propose a method for direct comparison of rendered images with a corresponding photograph in order to analyze the optical properties of physical objects and test the appropriateness of appearance models. To this end, we provide a practical method for aligning a known object and a point-like light source with the configuration observed in a photograph. Our method is based on projective transformation of object edges and silhouette matching in the image plane. To improve the similarity between rendered and photographed objects, we introduce models for spatially varying roughness and a model where the distribution of light transmitted by a rough surface influences direction-dependent subsurface scattering. Our goal is to support development toward progressive refinement of appearance models through quantitative validation.

Scene reassembly after multimodal digitization and pipeline evaluation using photorealistic rendering.

Transparent objects require acquisition modalities that are very different from the ones used for objects with more diffuse reflectance properties. Digitizing a scene where objects must be acquired with different modalities requires scene reassembly after reconstruction of the object surfaces. This reassembly of a scene that was picked apart for scanning seems unexplored. We contribute with a multimodal digitization pipeline for scenes that require this step of reassembly. Our pipeline includes measurement of bidirectional reflectance distribution functions and high dynamic range imaging of the lighting environment. This enables pixelwise comparison of photographs of the real scene with renderings of the digital version of the scene. Such quantitative evaluation is useful for verifying acquired material appearance and reconstructed surface geometry, which is an important aspect of digital content creation. It is also useful for identifying and improving issues in the different steps of the pipeline. In this work, we use it to improve reconstruction, apply analysis by synthesis to estimate optical properties, and to develop our method for scene reassembly.

A unified approach to diffusion direction sensitive slice registration and 3-D DTI reconstruction from moving fetal brain anatomy.

This paper presents an approach to 3-D diffusion tensor image (DTI) reconstruction from multi-slice diffusion weighted (DW) magnetic resonance imaging acquisitions of the moving fetal brain. Motion scatters the slice measurements in the spatial and spherical diffusion domain with respect to the underlying anatomy. Previous image registration techniques have been described to estimate the between slice fetal head motion, allowing the reconstruction of 3D a diffusion estimate on a regular grid using interpolation. We propose Approach to Unified Diffusion Sensitive Slice Alignment and Reconstruction (AUDiSSAR) that explicitly formulates a process for diffusion direction sensitive DW-slice-to-DTI-volume alignment. This also incorporates image resolution modeling to iteratively deconvolve the effects of the imaging point spread function using the multiple views provided by thick slices acquired in different anatomical planes. The algorithm is implemented using a multi-resolution iterative scheme and multiple real and synthetic data are used to evaluate the performance of the technique. An accuracy experiment using synthetically created motion data of an adult head and an experiment using synthetic motion added to sedated fetal monkey dataset show a significant improvement in motion-trajectory estimation compared to current state-of-the-art approaches. The performance of the method is then evaluated on challenging but clinically typical in utero fetal scans of four different human cases, showing improved rendition of cortical anatomy and extraction of white matter tracts. While the experimental work focuses on DTI reconstruction (second-order tensor model), the proposed reconstruction framework can employ any 5-D diffusion volume model that can be represented by the spatial parameterizations of an orientation distribution function.

Adapting parcellation schemes to study fetal brain connectivity in serial imaging studies.

A crucial step in studying brain connectivity is the definition of the Regions Of Interest (ROI's) which are considered as nodes of a network graph. These ROI's identified in structural imaging reflect consistent functional regions in the anatomies being compared. However in serial studies of the developing fetal brain such functional and associated structural markers are not consistently present over time. In this study we adapt two non-atlas based parcellation schemes to study the development of connectivity networks of a fetal monkey brain using Diffusion Weighted Imaging techniques. Results demonstrate that the fetal brain network exhibits small-world characteristics and a pattern of increased cluster coefficients and decreased global efficiency. These findings may provide a route to creating a new biomarker for healthy fetal brain development.