Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model.

We show that in a common high-dimensional covariance model, the choice of loss function has a profound effect on optimal estimation. In an asymptotic framework based on the Spiked Covariance model and use of orthogonally invariant estimators, we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker η that acts elementwise on the sample eigenvalues. Indeed, to each loss function there corresponds a unique admissible eigenvalue shrinker η* dominating all other shrinkers. The shape of the optimal shrinker is determined by the choice of loss function and, crucially, by inconsistency of both eigenvalues and eigenvectors of the sample covariance matrix. Details of these phenomena and closed form formulas for the optimal eigenvalue shrinkers are worked out for a menagerie of 26 loss functions for covariance estimation found in the literature, including the Stein, Entropy, Divergence, Fréchet, Bhattacharya/Matusita, Frobenius Norm, Operator Norm, Nuclear Norm and Condition Number losses.

Tail sums of Wishart and Gaussian eigenvalues beyond the bulk edge.

Consider the classical Gaussian unitary ensemble of size N and the real white Wishart ensemble with N variables and n degrees of freedom. In the limits of large N and n, with positive ratio γ in the Wishart case, the expected number of eigenvalues that exit the upper bulk edge is less than one, approaching 0.031 and 0.170 respectively, the latter number being independent of γ. These statements are consequences of quantitative bounds on tail sums of eigenvalues outside the bulk which are established here for applications in high dimensional covariance matrix estimation.