The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem.

Deep learning (DL) has had unprecedented success and is now entering scientific computing with full force. However, current DL methods typically suffer from instability, even when universal approximation properties guarantee the existence of stable neural networks (NNs). We address this paradox by demonstrating basic well-conditioned problems in scientific computing where one can prove the existence of NNs with great approximation qualities; however, there does not exist any algorithm, even randomized, that can train (or compute) such a NN. For any positive integers K>2 and L, there are cases where simultaneously 1) no randomized training algorithm can compute a NN correct to K digits with probability greater than 1/2; 2) there exists a deterministic training algorithm that computes a NN with K ­1 correct digits, but any such (even randomized) algorithm needs arbitrarily many training data; and 3) there exists a deterministic training algorithm that computes a NN with K ­2 correct digits using no more than L training samples. These results imply a classification theory describing conditions under which (stable) NNs with a given accuracy can be computed by an algorithm. We begin this theory by establishing sufficient conditions for the existence of algorithms that compute stable NNs in inverse problems. We introduce fast iterative restarted networks (FIRENETs), which we both prove and numerically verify are stable. Moreover, we prove that only O(|log (ϵ)|) layers are needed for an ϵ-accurate solution to the inverse problem.

How to Compute Spectra with Error Control.

Computing the spectra of operators is a fundamental problem in the sciences, with wide-ranging applications in condensed-matter physics, quantum mechanics and chemistry, statistical mechanics, etc. While there are algorithms that in certain cases converge to the spectrum, no general procedure is known that (a) always converges, (b) provides bounds on the errors of approximation, and (c) provides approximate eigenvectors. This may lead to incorrect simulations. It has been an open problem since the 1950s to decide whether such reliable methods exist at all. We affirmatively resolve this question, and the algorithms provided are optimal, realizing the boundary of what digital computers can achieve. Moreover, they are easy to implement and parallelize, offer fundamental speed-ups, and allow problems that before, regardless of computing power, were out of reach. Results are demonstrated on difficult problems such as the spectra of quasicrystals and non-Hermitian phase transitions in optics.

A Mathieu function boundary spectral method for scattering by multiple variable poro-elastic plates, with applications to metamaterials and acoustics.

Many problems in fluid mechanics and acoustics can be modelled by Helmholtz scattering off poro-elastic plates. We develop a boundary spectral method, based on collocation of local Mathieu function expansions, for Helmholtz scattering off multiple variable poro-elastic plates in two dimensions. Such boundary conditions, namely the varying physical parameters and coupled thin-plate equation, present a considerable challenge to current methods. The new method is fast, accurate and flexible, with the ability to compute expansions in thousands (and even tens of thousands) of Mathieu functions, thus making it a favourable method for the considered geometries. Comparisons are made with elastic boundary element methods, where the new method is found to be faster and more accurate. Our solution representation directly provides a sine series approximation of the far-field directivity and can be evaluated near or on the scatterers, meaning that the near field can be computed stably and efficiently. The new method also allows us to examine the effects of varying stiffness along a plate, which is poorly studied due to limitations of other available techniques. We show that a power-law decrease to zero in stiffness parameters gives rise to unexpected scattering and aeroacoustic effects similar to an acoustic black hole metamaterial.

The unified transform for mixed boundary condition problems in unbounded domains.

This paper implements the unified transform to problems in unbounded domains with solutions having corner singularities. Consequently, a wide variety of mixed boundary condition problems can be solved without the need for the Wiener-Hopf technique. Such problems arise frequently in acoustic scattering or in the calculation of electric fields in geometries involving finite and/or multiple plates. The new approach constructs a global relation that relates known boundary data, such as the scattered normal velocity on a rigid plate, to unknown boundary values, such as the jump in pressure upstream of the plate. By approximating the known data and the unknown boundary values by suitable functions and evaluating the global relation at collocation points, one can accurately obtain the expansion coefficients of the unknown boundary values. The method is illustrated for the modified Helmholtz and Helmholtz equations. In each case, comparisons between the traditional Wiener-Hopf approach, other spectral or boundary methods and the unified transform approach are discussed.

Correction to 'The unified transform for mixed boundary condition problems in unbounded domains'.

[This corrects the article DOI: 10.1098/rspa.2018.0605.].