Detection and mapping of specular surfaces using multibounce LiDAR returns.

We propose methods that use specular, multibounce LiDAR returns to detect and map specular surfaces that might be invisible to conventional LiDAR systems that rely on direct, single-scatter returns. We derive expressions that relate the time- and angle-of-arrival of these multibounce returns to scattering points on the specular surface, and then use these expressions to formulate techniques for retrieving specular surface geometry when the scene is scanned by a single beam or illuminated with a multi-beam flash. We also consider the special case of transparent specular surfaces, for which surface reflections can be mixed together with light that scatters off of objects lying behind the surface.

Computer vision uncovers predictors of physical urban change.

Which neighborhoods experience physical improvements? In this paper, we introduce a computer vision method to measure changes in the physical appearances of neighborhoods from time-series street-level imagery. We connect changes in the physical appearance of five US cities with economic and demographic data and find three factors that predict neighborhood improvement. First, neighborhoods that are densely populated by college-educated adults are more likely to experience physical improvements-an observation that is compatible with the economic literature linking human capital and local success. Second, neighborhoods with better initial appearances experience, on average, larger positive improvements-an observation that is consistent with "tipping" theories of urban change. Third, neighborhood improvement correlates positively with physical proximity to the central business district and to other physically attractive neighborhoods-an observation that is consistent with the "invasion" theories of urban sociology. Together, our results provide support for three classical theories of urban change and illustrate the value of using computer vision methods and street-level imagery to understand the physical dynamics of cities.

Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy.

High-speed, large-scale three-dimensional (3D) imaging of neuronal activity poses a major challenge in neuroscience. Here we demonstrate simultaneous functional imaging of neuronal activity at single-neuron resolution in an entire Caenorhabditis elegans and in larval zebrafish brain. Our technique captures the dynamics of spiking neurons in volumes of ∼700 µm × 700 µm × 200 µm at 20 Hz. Its simplicity makes it an attractive tool for high-speed volumetric calcium imaging.

Object classification through scattering media with deep learning on time resolved measurement.

We demonstrate an imaging technique that allows identification and classification of objects hidden behind scattering media and is invariant to changes in calibration parameters within a training range. Traditional techniques to image through scattering solve an inverse problem and are limited by the need to tune a forward model with multiple calibration parameters (like camera field of view, illumination position etc.). Instead of tuning a forward model and directly inverting the optical scattering, we use a data driven approach and leverage convolutional neural networks (CNN) to learn a model that is invariant to calibration parameters variations within the training range and nearly invariant beyond that. This effectively allows robust imaging through scattering conditions that is not sensitive to calibration. The CNN is trained with a large synthetic dataset generated with a Monte Carlo (MC) model that contains random realizations of major calibration parameters. The method is evaluated with a time-resolved camera and multiple experimental results are provided including pose estimation of a mannequin hidden behind a paper sheet with 23 correct classifications out of 30 tests in three poses (76.6% accuracy on real-world measurements). This approach paves the way towards real-time practical non line of sight (NLOS) imaging applications.

Terahertz scattering and water absorption for porosimetry.

We use terahertz transmission through limestone sedimentary rock samples to assess the macro and micro porosity. We exploit the notable water absorption in the terahertz spectrum to interact with the pores that are two orders of magnitude smaller (<1µm) than the terahertz wavelength. Terahertz water sensitivity provides us with the dehydration profile of the rock samples. The results show that there is a linear correlation between such a profile and the ratio of micro to macro porosity of the rock. Furthermore, this study estimates the absolute value of total porosity based on optical diffusion theory. We compare our results with that of mercury injection capillary pressure as a benchmark to confirm our analytic framework. The porosimetry method presented here sets a foundation for a new generation of less invasive porosimetry methods with higher penetration depth based on lower frequency (f<10THz) scattering and absorption. The technique has applications in geological studies and in other industries without the need for hazardous mercury or ionizing radiation.

Modeling "wiggling" as a multi-path interference problem in AMCW ToF imaging.

Amplitude modulated continuous wave time-of-flight range cameras suffer from an inherent depth measurement error due to aliasing of the emitted signal vs reference signal correlation function. This is due to higher harmonics present in both signals which are not accounted for in the model or measurements. This "wiggling" error is generally corrected by employing a correction function based on frequency and depth dependent calibration data. This problem is shown to be equivalent to a multi-path interference problem. Casting the problem into the multi-path interference domain and utilizing multiple modulation frequencies provides tools for dealing with the depth error without calibration in a frequency independent way.

Attenuation-corrected fluorescence spectra unmixing for spectroscopy and microscopy.

In fluorescence measurements, light is often absorbed and scattered by a sample both for excitation and emission, resulting in the measured spectra to be distorted. Conventional linear unmixing methods computationally separate overlapping spectra but do not account for these effects. We propose a new algorithm for fluorescence unmixing that accounts for the attenuation-related distortion effect on fluorescence spectra. Using a matrix representation, we derive forward measurement formation and a corresponding inverse method; the unmixing algorithm is based on nonnegative matrix factorization. We also demonstrate how this method can be extended to a higher-dimensional tensor form, which is useful for unmixing overlapping spectra observed under the attenuation effect in spectral imaging microscopy. We evaluate the proposed methods in simulation and experiments and show that it outperforms a conventional, linear unmixing method when absorption and scattering contributes to the measured signals, as in deep tissue imaging.

Pose estimation using time-resolved inversion of diffuse light.

We present a novel approach for evaluation of position and orientation of geometric shapes from scattered time-resolved data. Traditionally, imaging systems treat scattering as unwanted and are designed to mitigate the effects. Instead, we show here that scattering can be exploited by implementing a system based on a femtosecond laser and a streak camera. The result is accurate estimation of object pose, which is a fundamental tool in analysis of complex scenarios and plays an important role in our understanding of physical phenomena. Here, we experimentally show that for a given geometry, a single incident illumination point yields enough information for pose estimation and tracking after multiple scattering events. Our technique can be used for single-shot imaging behind walls or through turbid media.

Compressive multi-mode superresolution display.

Compressive displays are an emerging technology exploring the co-design of new optical device configurations and compressive computation. Previously, research has shown how to improve the dynamic range of displays and facilitate high-quality light field or glasses-free 3D image synthesis. In this paper, we introduce a new multi-mode compressive display architecture that supports switching between 3D and high dynamic range (HDR) modes as well as a new super-resolution mode. The proposed hardware consists of readily-available components and is driven by a novel splitting algorithm that computes the pixel states from a target high-resolution image. In effect, the display pixels present a compressed representation of the target image that is perceived as a single, high resolution image.

Robust and accurate transient light transport decomposition via convolutional sparse coding.

Ultrafast sources and detectors have been used to record the time-resolved scattering of light propagating through macroscopic scenes. In the context of computational imaging, decomposition of this transient light transport (TLT) is useful for applications, such as characterizing materials, imaging through diffuser layers, and relighting scenes dynamically. Here, we demonstrate a method of convolutional sparse coding to decompose TLT into direct reflections, inter-reflections, and subsurface scattering. The method relies on the sparsity composition of the time-resolved kernel. We show that it is robust and accurate to noise during the acquisition process.

Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization.

Time-of-flight (ToF) cameras calculate depth maps by reconstructing phase shifts of amplitude-modulated signals. For broad illumination of transparent objects, reflections from multiple scene points can illuminate a given pixel, giving rise to an erroneous depth map. We report here a sparsity-regularized solution that separates K interfering components using multiple modulation frequency measurements. The method maps ToF imaging to the general framework of spectral estimation theory and has applications in improving depth profiles and exploiting multiple scattering.

Estimating wide-angle, spatially varying reflectance using time-resolved inversion of backscattered light.

Imaging through complex media is a well-known challenge, as scattering distorts a signal and invalidates imaging equations. For coherent imaging, the input field can be reconstructed using phase conjugation or knowledge of the complex transmission matrix. However, for incoherent light, wave interference methods are limited to small viewing angles. On the other hand, time-resolved methods do not rely on signal or object phase correlations, making them suitable for reconstructing wide-angle, larger-scale objects. Previously, a time-resolved technique was demonstrated for uniformly reflecting objects. Here, we generalize the technique to reconstruct the spatially varying reflectance of shapes hidden by angle-dependent diffuse layers. The technique is a noninvasive method of imaging three-dimensional objects without relying on coherence. For a given diffuser, ultrafast measurements are used in a convex optimization program to reconstruct a wide-angle, three-dimensional reflectance function. The method has potential use for biological imaging and material characterization.

Using neural networks to calibrate agent based models enables improved regional evidence for vaccine strategy and policy.

Distribution and administration strategy are critical to successful population immunization efforts. Agent-based modeling (ABM) can reflect the complexity of real-world populations and can experimentally evaluate vaccine strategy and policy. However, ABMs historically have been limited in their time-to-development, long runtime, and difficulty calibrating. Our team had several technical advances in the development of our GradABMs: a novel class of scalable, fast and differentiable simulations. GradABMs can simulate million-size populations in a few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous sources. This allows for rapid and real-world sensitivity analyses. Our first epidemiological GradABM (EpiABMv1) enabled simulation interventions over real million-scale populations and was used in vaccine strategy and policy during the COVID-19 pandemic. Literature suggests decisions aided by evidence from these models saved thousands of lives. Our most recent model (EpiABMv2) extends EpiABMv1 to allow improved regional calibration using deep neural networks to incorporate local population data, and in some cases different policy recommendations versus our prior models. This is an important advance for our model to be more effective at vaccine strategy and policy decisions at the local public health level.

Reconstruction of hidden 3D shapes using diffuse reflections.

We analyze multi-bounce propagation of light in an unknown hidden volume and demonstrate that the reflected light contains sufficient information to recover the 3D structure of the hidden scene. We formulate the forward and inverse theory of secondary scattering using ideas from energy front propagation and tomography. We show that using Fresnel approximation greatly simplifies this problem and the inversion can be achieved via a backpropagation process. We study the invertibility, uniqueness and choices of space-time-angle dimensions using synthetic examples. We show that a 2D streak camera can be used to discover and reconstruct hidden geometry. Using a 1D high speed time of flight camera, we show that our method can be used recover 3D shapes of objects "around the corner".

Public health impact of delaying second dose of BNT162b2 or mRNA-1273 covid-19 vaccine: simulation agent based modeling study.

OBJECTIVE: To estimate population health outcomes with delayed second dose versus standard schedule of SARS-CoV-2 mRNA vaccination. DESIGN: Simulation agent based modeling study. SETTING: Simulated population based on real world US county. PARTICIPANTS: The simulation included 100 000 agents, with a representative distribution of demographics and occupations. Networks of contacts were established to simulate potentially infectious interactions though occupation, household, and random interactions. INTERVENTIONS: Simulation of standard covid-19 vaccination versus delayed second dose vaccination prioritizing the first dose. The simulation runs were replicated 10 times. Sensitivity analyses included first dose vaccine efficacy of 50%, 60%, 70%, 80%, and 90% after day 12 post-vaccination; vaccination rate of 0.1%, 0.3%, and 1% of population per day; assuming the vaccine prevents only symptoms but not asymptomatic spread (that is, non-sterilizing vaccine); and an alternative vaccination strategy that implements delayed second dose for people under 65 years of age, but not until all those above this age have been vaccinated. MAIN OUTCOME MEASURES: Cumulative covid-19 mortality, cumulative SARS-CoV-2 infections, and cumulative hospital admissions due to covid-19 over 180 days. RESULTS: Over all simulation replications, the median cumulative mortality per 100 000 for standard dosing versus delayed second dose was 226 v 179, 233 v 207, and 235 v 236 for 90%, 80%, and 70% first dose efficacy, respectively. The delayed second dose strategy was optimal for vaccine efficacies at or above 80% and vaccination rates at or below 0.3% of the population per day, under both sterilizing and non-sterilizing vaccine assumptions, resulting in absolute cumulative mortality reductions between 26 and 47 per 100 000. The delayed second dose strategy for people under 65 performed consistently well under all vaccination rates tested. CONCLUSIONS: A delayed second dose vaccination strategy, at least for people aged under 65, could result in reduced cumulative mortality under certain conditions.

Iterative aperture mask design in phase space using a rank constraint.

We present an iterative camera aperture design procedure, which determines an optimal mask pattern based on a sparse set of desired intensity distributions at different focal depths. This iterative method uses the ambiguity function as a tool to shape the camera's response to defocus, and shares conceptual similarities with phase retrieval procedures. An analysis of algorithm convergence is presented, and experimental examples are shown to demonstrate the flexibility of the design process. This algorithm potentially ties together previous disjointed PSF design approaches under a common framework, and offers new insights for the creation of future application-specific imaging systems.

Universal Shelter-in-Place Versus Advanced Automated Contact Tracing and Targeted Isolation: A Case for 21st-Century Technologies for SARS-CoV-2 and Future Pandemics.

OBJECTIVE: To model and compare effect of digital contact tracing versus shelter-in-place on severe acute respiratory syndrome - coronavirus 2 (SARS-CoV-2) spread. METHODS: Using a classical epidemiologic framework and parameters estimated from literature published between February 1, 2020, and May 25, 2020, we modeled two non-pharmacologic interventions - shelter-in-place and digital contact tracing - to curb spread of SARS-CoV-2. For contact tracing, we assumed an advanced automated contact tracing (AACT) application that sends alerts to individuals advising self-isolation based on individual exposure profile. Model parameters included percentage population ordered to shelter-in-place, adoption rate of AACT, and percentage individuals who appropriately follow recommendations. Under influence of these variables, the number of individuals infected, exposed, and isolated were estimated. RESULTS: Without any intervention, a high rate of infection (>10 million) with early peak is predicted. Shelter-in-place results in rapid decline in infection rate at the expense of impacting a large population segment. The AACT model achieves reduction in infected and exposed individuals similar to shelter-in-place without impacting a large number of individuals. For example, a 50% AACT adoption rate mimics a shelter-in-place order for 40% of the population and results in a greater than 90% decrease in peak number of infections. However, as compared to shelter-in-place, with AACT significantly fewer individuals would be isolated. CONCLUSION: Wide adoption of digital contact tracing can mitigate infection spread similar to universal shelter-in-place, but with considerably fewer individuals isolated.

Multiflash stereopsis: depth-edge-preserving stereo with small baseline illumination.

Traditional stereo matching algorithms are limited in their ability to produce accurate results near depth discontinuities, due to partial occlusions and violation of smoothness constraints. In this paper, we use small baseline multi-flash illumination to produce a rich set of feature maps that enable acquisition of discontinuity preserving point correspondences. First, from a single multi-flash camera, we formulate a qualitative depth map using a gradient domain method that encodes object relative distances. Then, in a multiview setup, we exploit shadows created by light sources to compute an occlusion map. Finally, we demonstrate the usefulness of these feature maps by incorporating them into two different dense stereo correspondence algorithms, the first based on local search and the second based on belief propagation. Experimental results show that our enhanced stereo algorithms are able to extract high quality, discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods. We also demonstrate that small baseline illumination can be useful to handle specular reflections in stereo imagery. Different from most existing active illumination techniques, our method is simple, inexpensive, compact, and requires no calibration of light sources.

Machine learning approaches for large scale classification of produce.

The analysis and identification of different attributes of produce such as taxonomy, vendor, and organic nature is vital to verifying product authenticity in a distribution network. Though a variety of analysis techniques have been studied in the past, we present a novel data-centric approach to classifying produce attributes. We employed visible and near infrared (NIR) spectroscopy on over 75,000 samples across several fruit and vegetable varieties. This yielded 0.90-0.98 and 0.98-0.99 classification accuracies for taxonomy and farmer classes, respectively. The most significant factors in the visible spectrum were variations in the produce color due to chlorophyll and anthocyanins. In the infrared spectrum, we observed that the varying water and sugar content levels were critical to obtaining high classification accuracies. High quality spectral data along with an optimal tuning of hyperparameters in the support vector machine (SVM) was also key to achieving high classification accuracies. In addition to demonstrating exceptional accuracies on test data, we explored insights behind the classifications, and identified the highest performing approaches using cross validation. We presented data collection guidelines, experimental design parameters, and machine learning optimization parameters for the replication of studies involving large sample sizes.

Author Correction: Active Printed Materials for Complex Self-Evolving Deformations.

A correction has been published and is appended to both the HTML and PDF versions of this paper. The error has not been fixed in the paper.