On cheap entropy-sparsified regression learning.

Regression learning is one of the long-standing problems in statistics, machine learning, and deep learning (DL). We show that writing this problem as a probabilistic expectation over (unknown) feature probabilities - thus increasing the number of unknown parameters and seemingly making the problem more complex-actually leads to its simplification, and allows incorporating the physical principle of entropy maximization. It helps decompose a very general setting of this learning problem (including discretization, feature selection, and learning multiple piece-wise linear regressions) into an iterative sequence of simple substeps, which are either analytically solvable or cheaply computable through an efficient second-order numerical solver with a sublinear cost scaling. This leads to the computationally cheap and robust non-DL second-order Sparse Probabilistic Approximation for Regression Task Analysis (SPARTAn) algorithm, that can be efficiently applied to problems with millions of feature dimensions on a commodity laptop, when the state-of-the-art learning tools would require supercomputers. SPARTAn is compared to a range of commonly used regression learning tools on synthetic problems and on the prediction of the El Niño Southern Oscillation, the dominant interannual mode of tropical climate variability. The obtained SPARTAn learners provide more predictive, sparse, and physically explainable data descriptions, clearly discerning the important role of ocean temperature variability at the thermocline in the equatorial Pacific. SPARTAn provides an easily interpretable description of the timescales by which these thermocline temperature features evolve and eventually express at the surface, thereby enabling enhanced predictability of the key drivers of the interannual climate.

eSPA+: Scalable Entropy-Optimal Machine Learning Classification for Small Data Problems.

Classification problems in the small data regime (with small data statistic T and relatively large feature space dimension D) impose challenges for the common machine learning (ML) and deep learning (DL) tools. The standard learning methods from these areas tend to show a lack of robustness when applied to data sets with significantly fewer data points than dimensions and quickly reach the overfitting bound, thus leading to poor performance beyond the training set. To tackle this issue, we propose eSPA+, a significant extension of the recently formulated entropy-optimal scalable probabilistic approximation algorithm (eSPA). Specifically, we propose to change the order of the optimization steps and replace the most computationally expensive subproblem of eSPA with its closed-form solution. We prove that with these two enhancements, eSPA+ moves from the polynomial to the linear class of complexity scaling algorithms. On several small data learning benchmarks, we show that the eSPA+ algorithm achieves a many-fold speed-up with respect to eSPA and even better performance results when compared to a wide array of ML and DL tools. In particular, we benchmark eSPA+ against the standard eSPA and the main classes of common learning algorithms in the small data regime: various forms of support vector machines, random forests, and long short-term memory algorithms. In all the considered applications, the common learning methods and eSPA are markedly outperformed by eSPA+, which achieves significantly higher prediction accuracy with an orders-of-magnitude lower computational cost.

Systematic calculation of finite-time mixed singular vectors and characterization of error growth for persistent coherent atmospheric disturbances over Eurasia.

Singular vectors (SVs) have long been employed in the initialization of ensemble numerical weather prediction (NWP) in order to capture the structural organization and growth rates of those perturbations or "errors" associated with initial condition errors and instability processes of the large scale flow. Due to their (super) exponential growth rates and spatial scales, initial SVs are typically combined empirically with evolved SVs in order to generate forecast perturbations whose structures and growth rates are tuned for specified lead-times. Here, we present a systematic approach to generating finite time or "mixed" SVs (MSVs) based on a method for the calculation of covariant Lyapunov vectors and appropriate choices of the matrix cocycle. We first derive a data-driven reduced-order model to characterize persistent geopotential height anomalies over Europe and Western Asia (Eurasia) over the period 1979-present from the National Centers for Environmental Prediction v1 reanalysis. We then characterize and compare the MSVs and SVs of each persistent state over Eurasia for particular lead-times from a day to over a week. Finally, we compare the spatiotemporal properties of SVs and MSVs in an examination of the dynamics of the 2010 Russian heatwave. We show that MSVs provide a systematic approach to generate initial forecast perturbations projected onto relevant expanding directions in phase space for typical NWP forecast lead-times.

Intrinsic processes drive variability in basal melting of the Totten Glacier Ice Shelf.

Over the period 2003-2008, the Totten Ice Shelf (TIS) was shown to be rapidly thinning, likely due to basal melting. However, a recent study using a longer time series found high interannual variability present in TIS surface elevation without any apparent trend. Here we show that low-frequency intrinsic ocean variability potentially accounts for a large fraction of the variability in the basal melting of TIS. Specifically, numerical ocean model simulations show that up to 44% of the modelled variability in basal melting in the 1-5 year timescale (and up to 21% in the 5-10 year timescale) is intrinsic, with a similar response to the full climate forcing. We identify the important role of intrinsic ocean variability in setting the observed interannual variation in TIS surface thickness and velocity. Our results further demonstrate the need to account for intrinsic ocean processes in the detection and attribution of change.

Global patterns of change and variation in sea surface temperature and chlorophyll a.

Changes over the scale of decades in oceanic environments present a range of challenges for management and utilisation of ocean resources. Here we investigate sources of global temporal variation in Sea Surface Temperature (SST) and Ocean Colour (Chl-a) and their co-variation, over a 14 year period using statistical methodologies that partition sources of variation into inter-annual and annual components and explicitly account for daily auto-correlation. The variation in SST shows bands of increasing variability with increasing latitude, while the analysis of annual variability in Chl-a shows mostly mid-latitude high variability bands. Covariation patterns of SST and Chl-a suggests several different mechanisms impacting Chl-a change and variance. Our high spatial resolution analysis indicates these are likely to be operating at relatively small spatial scales. There are large regions showing warming and rising of Chl-a, contrasting with regions that show warming and decreasing Chl-a. The covariation pattern in annual variation in SST and Chl-a reveals broad latitudinal bands. On smaller scales there are significant regional anomalies where upwellings are known to occur. Over decadal time scales both trend and variation in SST, Chl-a and their covariance is highly spatially heterogeneous, indicating that monitoring and resource management must be regionally appropriate.

Sources of heterogeneous variability and trends in Antarctic sea-ice.

While the Northern Hemisphere sea-ice has uniformly declined over the past several decades, the observed sea-ice in the Southern Hemisphere has exhibited regions of increase and decrease. Here we use a comprehensive set of ocean-sea-ice simulations (1990-2007) to elucidate the drivers of the observed heterogeneous sea-ice trends. We show wind variability is an important determinant of the heterogeneous pattern of the variability and trends in Southern Hemisphere sea-ice. Only in the West Pacific region does Southern Annular Mode wind forcing contribute significantly to the trend in sea-ice duration. El Niño Southern Oscillation wind forcing contribution to the sea-ice duration trend is confined to the Atlantic and Pacific. In the Indian Ocean, weather is a significant driver of the sea-ice duration trend. Only in the East Pacific region is wind forcing alone insufficient to give rise to the observed sea-ice decline and must be augmented by warming to reproduce the observations.

Subgrid modelling for geophysical flows.

Recently developed closure-based and stochastic model approaches to subgrid-scale modelling of eddy interactions are reviewed. It is shown how statistical dynamical closure models can be used to self-consistently calculate the eddy damping and stochastic backscatter parameters, required in large eddy simulations (LESs), from higher resolution simulations. A closely related direct stochastic modelling scheme that is more generally applicable to complex models is then described and applied to LESs of quasi-geostrophic turbulence of the atmosphere and oceans. The fundamental differences between atmospheric and oceanic LESs, which are related to the difference in the deformation scales in the two classes of flows, are discussed. It is noted that a stochastic approach may be crucial when baroclinic instability is inadequately resolved. Finally, inhomogeneous closure theory is applied to the complex problem of flow over topography; it is shown that it can be used to understand the successes and limitations of currently used heuristic schemes and to provide a basis for further developments in the future.