Can contrastive learning avoid shortcut solutions?

The generalization of representations learned via contrastive learning depends crucially on what features of the data are extracted. However, we observe that the contrastive loss does not always sufficiently guide which features are extracted, a behavior that can negatively impact the performance on downstream tasks via "shortcuts", i.e., by inadvertently suppressing important predictive features. We find that feature extraction is influenced by the difficulty of the so-called instance discrimination task (i.e., the task of discriminating pairs of similar points from pairs of dissimilar ones). Although harder pairs improve the representation of some features, the improvement comes at the cost of suppressing previously well represented features. In response, we propose implicit feature modification (IFM), a method for altering positive and negative samples in order to guide contrastive models towards capturing a wider variety of predictive features. Empirically, we observe that IFM reduces feature suppression, and as a result improves performance on vision and medical imaging tasks. The code is available at: https://github.com/joshr17/IFM.

An Interpretable Predictive Model of Vaccine Utilization for Tanzania.

Providing accurate utilization forecasts is key to maintaining optimal vaccine stocks in any health facility. Current approaches to vaccine utilization forecasting are based on often outdated population census data, and rely on weak, low-dimensional demand forecasting models. Further, these models provide very little insights into factors that influence vaccine utilization. Here, we built a state-of-the-art, machine learning model using novel, temporally and regionally relevant vaccine utilization data. This highly multidimensional machine learning approach accurately predicted bi-weekly vaccine utilization at the individual health facility level. Specifically, we achieved a forecasting fraction error of less than two for about 45% of regional health facilities in both the Tanzania regions analyzed. Our "random forest regressor" had an average forecasting fraction error that was almost 18 times less compared to the existing system. Importantly, using our model, we gleaned several key insights into factors underlying utilization forecasts. This work serves as an important starting point to reimagining predictive health systems in the developing world by leveraging the power of Artificial Intelligence and big data.

Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices.

Data encoded as symmetric positive definite (SPD) matrices frequently arise in many areas of computer vision and machine learning. While these matrices form an open subset of the Euclidean space of symmetric matrices, viewing them through the lens of non-Euclidean Riemannian (Riem) geometry often turns out to be better suited in capturing several desirable data properties. Inspired by the great success of dictionary learning and sparse coding (DLSC) for vector-valued data, our goal in this paper is to represent data in the form of SPD matrices as sparse conic combinations of SPD atoms from a learned dictionary via a Riem geometric approach. To that end, we formulate a novel Riem optimization objective for DLSC, in which the representation loss is characterized via the affine-invariant Riem metric. We also present a computationally simple algorithm for optimizing our model. Experiments on several computer vision data sets demonstrate superior classification and retrieval performance using our approach when compared with SC via alternative non-Riem formulations.

Efficient nearest neighbors via robust sparse hashing.

This paper presents a new nearest neighbor (NN) retrieval framework: robust sparse hashing (RSH). Our approach is inspired by the success of dictionary learning for sparse coding. Our key idea is to sparse code the data using a learned dictionary, and then to generate hash codes out of these sparse codes for accurate and fast NN retrieval. But, direct application of sparse coding to NN retrieval poses a technical difficulty: when data are noisy or uncertain (which is the case with most real-world data sets), for a query point, an exact match of the hash code generated from the sparse code seldom happens, thereby breaking the NN retrieval. Borrowing ideas from robust optimization theory, we circumvent this difficulty via our novel robust dictionary learning and sparse coding framework called RSH, by learning dictionaries on the robustified counterparts of the perturbed data points. The algorithm is applied to NN retrieval on both simulated and real-world data. Our results demonstrate that RSH holds significant promise for efficient NN retrieval against the state of the art.

Jensen-Bregman LogDet divergence with application to efficient similarity search for covariance matrices.

Covariance matrices have found success in several computer vision applications, including activity recognition, visual surveillance, and diffusion tensor imaging. This is because they provide an easy platform for fusing multiple features compactly. An important task in all of these applications is to compare two covariance matrices using a (dis)similarity function, for which the common choice is the Riemannian metric on the manifold inhabited by these matrices. As this Riemannian manifold is not flat, the dissimilarities should take into account the curvature of the manifold. As a result, such distance computations tend to slow down, especially when the matrix dimensions are large or gradients are required. Further, suitability of the metric to enable efficient nearest neighbor retrieval is an important requirement in the contemporary times of big data analytics. To alleviate these difficulties, this paper proposes a novel dissimilarity measure for covariances, the Jensen-Bregman LogDet Divergence (JBLD). This divergence enjoys several desirable theoretical properties and at the same time is computationally less demanding (compared to standard measures). Utilizing the fact that the square root of JBLD is a metric, we address the problem of efficient nearest neighbor retrieval on large covariance datasets via a metric tree data structure. To this end, we propose a K-Means clustering algorithm on JBLD. We demonstrate the superior performance of JBLD on covariance datasets from several computer vision applications.