Search Results for author:
Found 46 papers, 22 papers with code
Efficient kernelized bandit algorithms via exploration distributions
We consider a kernelized bandit problem with a compact arm set ${X} \subset \mathbb{R}^d $ and a fixed but unknown reward function $f^*$ with a finite norm in some Reproducing Kernel Hilbert Space (RKHS).
On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization
Reinforcement learning (RL) has become popular in enhancing the reasoning capabilities of large language models (LLMs), with Group Relative Policy Optimization (GRPO) emerging as a widely used algorithm in recent systems.
Uncertainty Herding: One Active Learning Method for All Label Budgets
Most active learning research has focused on methods which perform well when many labels are available, but can be dramatically worse than random selection when label budgets are small.
Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics
Obtaining compositional mappings is important for the model to generalize well compositionally.
Learning Deep Kernels for Non-Parametric Independence Testing
The Hilbert-Schmidt Independence Criterion (HSIC) is a powerful tool for nonparametric detection of dependence between random variables.
Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition
We present a theoretical explanation of the ``grokking'' phenomenon, where a model generalizes long after overfitting, for the originally-studied problem of modular addition.
Generalized Coverage for More Robust Low-Budget Active Learning
The ProbCover method of Yehuda et al. is a well-motivated algorithm for active learning in low-budget regimes, which attempts to "cover" the data distribution with balls of a given radius at selected data points.
Learning Dynamics of LLM Finetuning
Learning dynamics, which describes how the learning of specific training examples influences the model's predictions on other examples, gives us a powerful tool for understanding the behavior of deep learning systems.
Bias Amplification in Language Model Evolution: An Iterated Learning Perspective
With the widespread adoption of Large Language Models (LLMs), the prevalence of iterative interactions among these models is anticipated to increase.
Practical Kernel Tests of Conditional Independence
We describe a data-efficient, kernel-based approach to statistical testing of conditional independence.
Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling
Most meta-learning methods assume that the (very small) context set used to establish a new task at test time is passively provided.
AdaFlood: Adaptive Flood Regularization
Although neural networks are conventionally optimized towards zero training loss, it has been recently learned that targeting a non-zero training loss threshold, referred to as a flood level, often enables better test time generalization.
Queer In AI: A Case Study in Community-Led Participatory AI
We present Queer in AI as a case study for community-led participatory design in AI.
Exphormer: Sparse Transformers for Graphs
We show that incorporating Exphormer into the recently-proposed GraphGPS framework produces models with competitive empirical results on a wide variety of graph datasets, including state-of-the-art results on three datasets.
Ranked #2 on
Link Prediction
on PCQM-Contact
(MRR metric)
Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation
Maximum mean discrepancy (MMD) is a particularly useful distance metric for differentially private data generation: when used with finite-dimensional features it allows us to summarize and privatize the data distribution once, which we can repeatedly use during generator training without further privacy loss.
How to prepare your task head for finetuning
We identify a significant trend in the effect of changes in this initial energy on the resulting features after fine-tuning.
Efficient Conditionally Invariant Representation Learning
The procedure requires just a single ridge regression from $Y$ to kernelized features of $Z$, which can be done in advance.
MMD-B-Fair: Learning Fair Representations with Statistical Testing
We introduce a method, MMD-B-Fair, to learn fair representations of data via kernel two-sample testing.
A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models
We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau envelopes of the loss $\ell$.
Object Discovery via Contrastive Learning for Weakly Supervised Object Detection
Weakly Supervised Object Detection (WSOD) is a task that detects objects in an image using a model trained only on image-level annotations.
Ranked #1 on
Weakly Supervised Object Detection
on MS-COCO-2017
A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernel
Empirical neural tangent kernels (eNTKs) can provide a good understanding of a given network's representation: they are often far less expensive to compute and applicable more broadly than infinite width NTKs.
Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels
We propose a new method for approximating active learning acquisition strategies that are based on retraining with hypothetically-labeled candidate data points.
Evaluating Graph Generative Models with Contrastively Learned Features
A wide range of models have been proposed for Graph Generative Models, necessitating effective methods to evaluate their quality.
Pre-trained Perceptual Features Improve Differentially Private Image Generation
Training even moderately-sized generative models with differentially-private stochastic gradient descent (DP-SGD) is difficult: the required level of noise for reasonable levels of privacy is simply too large.
One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model
Semi-weakly supervised semantic segmentation (SWSSS) aims to train a model to identify objects in images based on a small number of images with pixel-level labels, and many more images with only image-level labels.
Better Supervisory Signals by Observing Learning Paths
Observing the learning path not only provides a new perspective for understanding knowledge distillation, overfitting, and learning dynamics, but also reveals that the supervisory signal of a teacher network can be very unstable near the best points in training on real tasks.
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression
We study a localized notion of uniform convergence known as an "optimistic rate" (Panchenko 2002; Srebro et al. 2010) for linear regression with Gaussian data.
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting
We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the class's Gaussian width.
Self-Supervised Learning with Kernel Dependence Maximization
We approach self-supervised learning of image representations from a statistical dependence perspective, proposing Self-Supervised Learning with the Hilbert-Schmidt Independence Criterion (SSL-HSIC).
Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data
In realistic scenarios with very limited numbers of data samples, however, it can be challenging to identify a kernel powerful enough to distinguish complex distributions.
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds and Benign Overfitting
We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the class’s Gaussian width.
Does Invariant Risk Minimization Capture Invariance?
We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al. (2019) can fail to capture "natural" invariances, at least when used in its practical "linear" form, and even on very simple problems which directly follow the motivating examples for IRM.
On Uniform Convergence and Low-Norm Interpolation Learning
But we argue we can explain the consistency of the minimal-norm interpolator with a slightly weaker, yet standard, notion: uniform convergence of zero-error predictors in a norm ball.
Learning Deep Kernels for Non-Parametric Two-Sample Tests
We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution.
Ranked #1 on
Two-sample testing
on HIGGS Data Set
Unbiased estimators for the variance of MMD estimators
The maximum mean discrepancy (MMD) is a kernel-based distance between probability distributions useful in many applications (Gretton et al. 2012), bearing a simple estimator with pleasing computational and statistical properties.
Learning deep kernels for exponential family densities
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching.
On gradient regularizers for MMD GANs
We propose a principled method for gradient-based regularization of the critic of GAN-like models trained by adversarially optimizing the kernel of a Maximum Mean Discrepancy (MMD).
Demystifying MMD GANs
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs.
Efficient and principled score estimation with Nyström kernel exponential families
We propose a fast method with statistical guarantees for learning an exponential family density model where the natural parameter is in a reproducing kernel Hilbert space, and may be infinite-dimensional.
Bayesian Approaches to Distribution Regression
Distribution regression has recently attracted much interest as a generic solution to the problem of supervised learning where labels are available at the group level, rather than at the individual level.
Fixing an error in Caponnetto and de Vito (2007)
The seminal paper of Caponnetto and de Vito (2007) provides minimax-optimal rates for kernel ridge regression in a very general setting.
Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
In this context, the MMD may be used in two roles: first, as a discriminator, either directly on the samples, or on features of the samples.
Understanding the 2016 US Presidential Election using ecological inference and distribution regression with census microdata
We combine fine-grained spatially referenced census data with the vote outcomes from the 2016 US presidential election.
Deep Mean Maps
The use of distributions and high-level features from deep architecture has become commonplace in modern computer vision.
Linear-time Learning on Distributions with Approximate Kernel Embeddings
This work develops the first random features for pdfs whose dot product approximates kernels using these non-Euclidean metrics, allowing estimators using such kernels to scale to large datasets by working in a primal space, without computing large Gram matrices.
Kernels on Sample Sets via Nonparametric Divergence Estimates
Most machine learning algorithms, such as classification or regression, treat the individual data point as the object of interest.
