Search Results for author:
Found 11 papers, 5 papers with code
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data.
Diagnosing and Rectifying Vision Models using Language
Our proposed method can discover high-error data slices, identify influential attributes and further rectify undesirable model behaviors, without requiring any visual data.
A Theoretical Study of Inductive Biases in Contrastive Learning
Understanding self-supervised learning is important but challenging.
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations
In particular, a linear classifier trained to separate the representations on the source domain can also predict classes on the target domain accurately, even though the representations of the two domains are far from each other.
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation
We consider unsupervised domain adaptation (UDA), where labeled data from a source domain (e. g., photographs) and unlabeled data from a target domain (e. g., sketches) are used to learn a classifier for the target domain.
Amortized Proximal Optimization
Using APO to adapt a structured preconditioning matrix generally results in optimization performance competitive with second-order methods.
Self-supervised Learning is More Robust to Dataset Imbalance
Third, inspired by the theoretical insights, we devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets with several evaluation criteria, closing the small gap between balanced and imbalanced datasets with the same number of examples.
Ranked #9 on
Long-tail Learning
on CIFAR-10-LT (ρ=100)
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Despite the empirical successes, theoretical foundations are limited -- prior analyses assume conditional independence of the positive pairs given the same class label, but recent empirical applications use heavily correlated positive pairs (i. e., data augmentations of the same image).
Meta-learning Transferable Representations with a Single Target Domain
Recent works found that fine-tuning and joint training---two popular approaches for transfer learning---do not always improve accuracy on downstream tasks.
Shape Matters: Understanding the Implicit Bias of the Noise Covariance
We show that in an over-parameterized setting, SGD with label noise recovers the sparse ground-truth with an arbitrary initialization, whereas SGD with Gaussian noise or gradient descent overfits to dense solutions with large norms.
Random Shuffling Beats SGD after Finite Epochs
We present the first (to our knowledge) non-asymptotic solution to this problem, which shows that after a "reasonable" number of epochs RandomShuffle indeed converges faster than SGD.
