In-context operator learning with data prompts for differential equation problems.

This paper introduces the paradigm of "in-context operator learning" and the corresponding model "In-Context Operator Networks" to simultaneously learn operators from the prompted data and apply it to new questions during the inference stage, without any weight update. Existing methods are limited to using a neural network to approximate a specific equation solution or a specific operator, requiring retraining when switching to a new problem with different equations. By training a single neural network as an operator learner, rather than a solution/operator approximator, we can not only get rid of retraining (even fine-tuning) the neural network for new problems but also leverage the commonalities shared across operators so that only a few examples in the prompt are needed when learning a new operator. Our numerical results show the capability of a single neural network as a few-shot operator learner for a diversified type of differential equation problems, including forward and inverse problems of ordinary differential equations, partial differential equations, and mean-field control problems, and also show that it can generalize its learning capability to operators beyond the training distribution.

Three-dimensional topological magnetic monopoles and their interactions in a ferromagnetic meta-lattice.

Topological magnetic monopoles (TMMs), also known as hedgehogs or Bloch points, are three-dimensional (3D) non-local spin textures that are robust to thermal and quantum fluctuations due to the topology protection1-4. Although TMMs have been observed in skyrmion lattices1,5, spinor Bose-Einstein condensates6,7, chiral magnets8, vortex rings2,9 and vortex cores10, it has been difficult to directly measure the 3D magnetization vector field of TMMs and probe their interactions at the nanoscale. Here we report the creation of 138 stable TMMs at the specific sites of a ferromagnetic meta-lattice at room temperature. We further develop soft X-ray vector ptycho-tomography to determine the magnetization vector and emergent magnetic field of the TMMs with a 3D spatial resolution of 10 nm. This spatial resolution is comparable to the magnetic exchange length of transition metals11, enabling us to probe monopole-monopole interactions. We find that the TMM and anti-TMM pairs are separated by 18.3 ± 1.6 nm, while the TMM and TMM, and anti-TMM and anti-TMM pairs are stabilized at comparatively longer distances of 36.1 ± 2.4 nm and 43.1 ± 2.0 nm, respectively. We also observe virtual TMMs created by magnetic voids in the meta-lattice. This work demonstrates that ferromagnetic meta-lattices could be used as a platform to create and investigate the interactions and dynamics of TMMs. Furthermore, we expect that soft X-ray vector ptycho-tomography can be broadly applied to quantitatively image 3D vector fields in magnetic and anisotropic materials at the nanoscale.

Extraction of Hidden Science from Nanoscale Images.

Scanning probe microscopies and spectroscopies enable investigation of surfaces and even buried interfaces down to the scale of chemical-bonding interactions, and this capability has been enhanced with the support of computational algorithms for data acquisition and image processing to explore physical, chemical, and biological phenomena. Here, we describe how scanning probe techniques have been enhanced by some of these recent algorithmic improvements. One improvement to the data acquisition algorithm is to advance beyond a simple rastering framework by using spirals at constant angular velocity then switching to constant linear velocity, which limits the piezo creep and hysteresis issues seen in traditional acquisition methods. One can also use image-processing techniques to model the distortions that appear from tip motion effects and to make corrections to these images. Another image-processing algorithm we discuss enables researchers to segment images by domains and subdomains, thereby highlighting reactive and interesting disordered sites at domain boundaries. Lastly, we discuss algorithms used to examine the dipole direction of individual molecules and surface domains, hydrogen bonding interactions, and molecular tilt. The computational algorithms used for scanning probe techniques are still improving rapidly and are incorporating machine learning at the next level of iteration. That said, the algorithms are not yet able to perform live adjustments during data recording that could enhance the microscopy and spectroscopic imaging methods significantly.

Three-dimensional atomic packing in amorphous solids with liquid-like structure.

Liquids and solids are two fundamental states of matter. However, our understanding of their three-dimensional atomic structure is mostly based on physical models. Here we use atomic electron tomography to experimentally determine the three-dimensional atomic positions of monatomic amorphous solids, namely a Ta thin film and two Pd nanoparticles. We observe that pentagonal bipyramids are the most abundant atomic motifs in these amorphous materials. Instead of forming icosahedra, the majority of pentagonal bipyramids arrange into pentagonal bipyramid networks with medium-range order. Molecular dynamics simulations further reveal that pentagonal bipyramid networks are prevalent in monatomic metallic liquids, which rapidly grow in size and form more icosahedra during the quench from the liquid to the glass state. These results expand our understanding of the atomic structures of amorphous solids and will encourage future studies on amorphous-crystalline phase and glass transitions in non-crystalline materials with three-dimensional atomic resolution.

Alternating the population and control neural networks to solve high-dimensional stochastic mean-field games.

We present APAC-Net, an alternating population and agent control neural network for solving stochastic mean-field games (MFGs). Our algorithm is geared toward high-dimensional instances of MFGs that are not approachable with existing solution methods. We achieve this in two steps. First, we take advantage of the underlying variational primal-dual structure that MFGs exhibit and phrase it as a convex-concave saddle-point problem. Second, we parameterize the value and density functions by two neural networks, respectively. By phrasing the problem in this manner, solving the MFG can be interpreted as a special case of training a generative adversarial network (GAN). We show the potential of our method on up to 100-dimensional MFG problems.

Determining the three-dimensional atomic structure of an amorphous solid.

Amorphous solids such as glass, plastics and amorphous thin films are ubiquitous in our daily life and have broad applications ranging from telecommunications to electronics and solar cells1-4. However, owing to the lack of long-range order, the three-dimensional (3D) atomic structure of amorphous solids has so far eluded direct experimental determination5-15. Here we develop an atomic electron tomography reconstruction method to experimentally determine the 3D atomic positions of an amorphous solid. Using a multi-component glass-forming alloy as proof of principle, we quantitatively characterize the short- and medium-range order of the 3D atomic arrangement. We observe that, although the 3D atomic packing of the short-range order is geometrically disordered, some short-range-order structures connect with each other to form crystal-like superclusters and give rise to medium-range order. We identify four types of crystal-like medium-range order-face-centred cubic, hexagonal close-packed, body-centred cubic and simple cubic-coexisting in the amorphous sample, showing translational but not orientational order. These observations provide direct experimental evidence to support the general framework of the efficient cluster packing model for metallic glasses10,12-14,16. We expect that this work will pave the way for the determination of the 3D structure of a wide range of amorphous solids, which could transform our fundamental understanding of non-crystalline materials and related phenomena.

Potential of Attosecond Coherent Diffractive Imaging.

Attosecond science has been transforming our understanding of electron dynamics in atoms, molecules, and solids. However, to date almost all of the attoscience experiments have been based on spectroscopic measurements because attosecond pulses have intrinsically very broad spectra due to the uncertainty principle and are incompatible with conventional imaging systems. Here we report an important advance towards achieving attosecond coherent diffractive imaging. Using simulated attosecond pulses, we simultaneously reconstruct the spectrum, 17 probes, and 17 spectral images of extended objects from a set of ptychographic diffraction patterns. We further confirm the principle and feasibility of this method by successfully performing a ptychographic coherent diffractive imaging experiment using a light-emitting diode with a broad spectrum. We believe this work clears the way to an unexplored domain of attosecond imaging science, which could have a far-reaching impact across different disciplines.

A machine learning framework for solving high-dimensional mean field game and mean field control problems.

Mean field games (MFG) and mean field control (MFC) are critical classes of multiagent models for the efficient analysis of massive populations of interacting agents. Their areas of application span topics in economics, finance, game theory, industrial engineering, crowd motion, and more. In this paper, we provide a flexible machine learning framework for the numerical solution of potential MFG and MFC models. State-of-the-art numerical methods for solving such problems utilize spatial discretization that leads to a curse of dimensionality. We approximately solve high-dimensional problems by combining Lagrangian and Eulerian viewpoints and leveraging recent advances from machine learning. More precisely, we work with a Lagrangian formulation of the problem and enforce the underlying Hamilton-Jacobi-Bellman (HJB) equation that is derived from the Eulerian formulation. Finally, a tailored neural network parameterization of the MFG/MFC solution helps us avoid any spatial discretization. Our numerical results include the approximate solution of 100-dimensional instances of optimal transport and crowd motion problems on a standard work station and a validation using a Eulerian solver in two dimensions. These results open the door to much-anticipated applications of MFG and MFC models that are beyond reach with existing numerical methods.

Mapping Buried Hydrogen-Bonding Networks.

We map buried hydrogen-bonding networks within self-assembled monolayers of 3-mercapto-N-nonylpropionamide on Au{111}. The contributing interactions include the buried S-Au bonds at the substrate surface and the buried plane of linear networks of hydrogen bonds. Both are simultaneously mapped with submolecular resolution, in addition to the exposed interface, to determine the orientations of molecular segments and directional bonding. Two-dimensional mode-decomposition techniques are used to elucidate the directionality of these networks. We find that amide-based hydrogen bonds cross molecular domain boundaries and areas of local disorder.

Normal Estimation of a Transparent Object Using a Video.

Reconstructing transparent objects is a challenging problem. While producing reasonable results for quite complex objects, existing approaches require custom calibration or somewhat expensive labor to achieve high precision. When an overall shape preserving salient and fine details is sufficient, we show in this paper a significant step toward solving the problem when the object's silhouette is available and simple user interaction is allowed, by using a video of a transparent object shot under varying illumination. Specifically, we estimate the normal map of the exterior surface of a given solid transparent object, from which the surface depth can be integrated. Our technical contribution lies in relating this normal estimation problem to one of graph-cut segmentation. Unlike conventional formulations, however, our graph is dual-layered, since we can see a transparent object's foreground as well as the background behind it. Quantitative and qualitative evaluation are performed to verify the efficacy of this practical solution.

Defect-Tolerant Aligned Dipoles within Two-Dimensional Plastic Lattices.

Carboranethiol molecules self-assemble into upright molecular monolayers on Au{111} with aligned dipoles in two dimensions. The positions and offsets of each molecule's geometric apex and local dipole moment are measured and correlated with sub-Ångström precision. Juxtaposing simultaneously acquired images, we observe monodirectional offsets between the molecular apexes and dipole extrema. We determine dipole orientations using efficient new image analysis techniques and find aligned dipoles to be highly defect tolerant, crossing molecular domain boundaries and substrate step edges. The alignment observed, consistent with Monte Carlo simulations, forms through favorable intermolecular dipole-dipole interactions.

Radiation dose reduction in medical x-ray CT via Fourier-based iterative reconstruction.

PURPOSE: A Fourier-based iterative reconstruction technique, termed Equally Sloped Tomography (EST), is developed in conjunction with advanced mathematical regularization to investigate radiation dose reduction in x-ray CT. The method is experimentally implemented on fan-beam CT and evaluated as a function of imaging dose on a series of image quality phantoms and anonymous pediatric patient data sets. Numerical simulation experiments are also performed to explore the extension of EST to helical cone-beam geometry. METHODS: EST is a Fourier based iterative algorithm, which iterates back and forth between real and Fourier space utilizing the algebraically exact pseudopolar fast Fourier transform (PPFFT). In each iteration, physical constraints and mathematical regularization are applied in real space, while the measured data are enforced in Fourier space. The algorithm is automatically terminated when a proposed termination criterion is met. Experimentally, fan-beam projections were acquired by the Siemens z-flying focal spot technology, and subsequently interleaved and rebinned to a pseudopolar grid. Image quality phantoms were scanned at systematically varied mAs settings, reconstructed by EST and conventional reconstruction methods such as filtered back projection (FBP), and quantified using metrics including resolution, signal-to-noise ratios (SNRs), and contrast-to-noise ratios (CNRs). Pediatric data sets were reconstructed at their original acquisition settings and additionally simulated to lower dose settings for comparison and evaluation of the potential for radiation dose reduction. Numerical experiments were conducted to quantify EST and other iterative methods in terms of image quality and computation time. The extension of EST to helical cone-beam CT was implemented by using the advanced single-slice rebinning (ASSR) method. RESULTS: Based on the phantom and pediatric patient fan-beam CT data, it is demonstrated that EST reconstructions with the lowest scanner flux setting of 39 mAs produce comparable image quality, resolution, and contrast relative to FBP with the 140 mAs flux setting. Compared to the algebraic reconstruction technique and the expectation maximization statistical reconstruction algorithm, a significant reduction in computation time is achieved with EST. Finally, numerical experiments on helical cone-beam CT data suggest that the combination of EST and ASSR produces reconstructions with higher image quality and lower noise than the Feldkamp Davis and Kress (FDK) method and the conventional ASSR approach. CONCLUSIONS: A Fourier-based iterative method has been applied to the reconstruction of fan-bean CT data with reduced x-ray fluence. This method incorporates advantageous features in both real and Fourier space iterative schemes: using a fast and algebraically exact method to calculate forward projection, enforcing the measured data in Fourier space, and applying physical constraints and flexible regularization in real space. Our results suggest that EST can be utilized for radiation dose reduction in x-ray CT via the readily implementable technique of lowering mAs settings. Numerical experiments further indicate that EST requires less computation time than several other iterative algorithms and can, in principle, be extended to helical cone-beam geometry in combination with the ASSR method.

Mathematical filtering minimizes metallic halation of titanium implants in MicroCT images.

Microcomputed tomography (MicroCT) images containing titanium implant suffer from x-rays scattering, artifact and the implant surface is critically affected by metallic halation. To improve the metallic halation artifact, a nonlinear Total Variation denoising algorithm such as Split Bregman algorithm was applied to the digital data set of MicroCT images. This study demonstrated that the use of a mathematical filter could successfully reduce metallic halation, facilitating the osseointegration evaluation at the bone implant interface in the reconstructed images.

FAST EDGE-FILTERED IMAGE UPSAMPLING.

We present a novel edge preserved interpolation scheme for fast upsampling of natural images. The proposed piecewise hyperbolic operator uses a slope-limiter function that conveniently lends itself to higher-order approximations and is responsible for restricting spatial oscillations arising due to the edges and sharp details in the image. As a consequence the upsampled image not only exhibits enhanced edges, and discontinuities across boundaries, but also preserves smoothly varying features in images. Experimental results show an improvement in the PSNR compared to typical cubic, and spline-based interpolation approaches.

Development and optimization of regularized tomographic reconstruction algorithms utilizing equally-sloped tomography.

We develop two new algorithms for tomographic reconstruction which incorporate the technique of equally-sloped tomography (EST) and allow for the optimized and flexible implementation of regularization schemes, such as total variation constraints, and the incorporation of arbitrary physical constraints. The founding structure of the developed algorithms is EST, a technique of tomographic acquisition and reconstruction first proposed by Miao in 2005 for performing tomographic image reconstructions from a limited number of noisy projections in an accurate manner by avoiding direct interpolations. EST has recently been successfully applied to coherent diffraction microscopy, electron microscopy, and computed tomography for image enhancement and radiation dose reduction. However, the bottleneck of EST lies in its slow speed due to its higher computation requirements. In this paper, we formulate the EST approach as a constrained problem and subsequently transform it into a series of linear problems, which can be accurately solved by the operator splitting method. Based on these mathematical formulations, we develop two iterative algorithms for tomographic image reconstructions through EST, which incorporate Bregman and continuative regularization. Our numerical experiment results indicate that the new tomographic image reconstruction algorithms not only significantly reduce the computational time, but also improve the image quality. We anticipate that EST coupled with the novel iterative algorithms will find broad applications in X-ray tomography, electron microscopy, coherent diffraction microscopy, and other tomography fields.

Comparing registration methods for mapping brain change using tensor-based morphometry.

Measures of brain changes can be computed from sequential MRI scans, providing valuable information on disease progression for neuroscientific studies and clinical trials. Tensor-based morphometry (TBM) creates maps of these brain changes, visualizing the 3D profile and rates of tissue growth or atrophy. In this paper, we examine the power of different nonrigid registration models to detect changes in TBM, and their stability when no real changes are present. Specifically, we investigate an asymmetric version of a recently proposed Unbiased registration method, using mutual information as the matching criterion. We compare matching functionals (sum of squared differences and mutual information), as well as large-deformation registration schemes (viscous fluid and inverse-consistent linear elastic registration methods versus Symmetric and Asymmetric Unbiased registration) for detecting changes in serial MRI scans of 10 elderly normal subjects and 10 patients with Alzheimer's Disease scanned at 2-week and 1-year intervals. We also analyzed registration results when matching images corrupted with artificial noise. We demonstrated that the unbiased methods, both symmetric and asymmetric, have higher reproducibility. The unbiased methods were also less likely to detect changes in the absence of any real physiological change. Moreover, they measured biological deformations more accurately by penalizing bias in the corresponding statistical maps.

MRI RESOLUTION ENHANCEMENT USING TOTAL VARIATION REGULARIZATION.

We propose a novel method for resolution enhancement for volumetric images based on a variational-based reconstruction approach. The reconstruction problem is posed using a deconvolution model that seeks to minimize the total variation norm of the image. Additionally, we propose a new edge-preserving operator that emphasizes and even enhances edges during the up-sampling and decimation of the image. The edge enhanced reconstruction is shown to yield significant improvement in resolution, especially preserving important edges containing anatomical information. This method is demonstrated as an enhancement tool for low-resolution, anisotropic, 3D brain MRI images, as well as a pre-processing step to improve skull-stripping segmentation of brain images.

Shape from defocus via diffusion.

Defocus can be modeled as a diffusion process and represented mathematically using the heat equation, where image blur corresponds to the diffusion of heat. This analogy can be extended to non-planar scenes by allowing a space-varying diffusion coefficient. The inverse problem of reconstructing 3-D structure from blurred images corresponds to an "inverse diffusion" that is notoriously ill-posed. We show how to bypass this problem by using the notion of relative blur. Given two images, within each neighborhood, the amount of diffusion necessary to transform the sharper image into the blurrier one depends on the depth of the scene. This can be used to devise a global algorithm to estimate the depth profile of the scene without recovering the deblurred image, using only forward diffusion.