Search Results for author:
Found 31 papers, 17 papers with code
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite-Sum Structure
Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions.
Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations
The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals.
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure
Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions.
Invariance and Stability of Deep Convolutional Representations
In this paper, we study deep signal representations that are near-invariant to groups of transformations and stable to the action of diffeomorphisms without losing signal information.
A Contextual Bandit Bake-off
Contextual bandit algorithms are essential for solving many real-world interactive machine learning problems.
A Kernel Perspective for Regularizing Deep Neural Networks
We propose a new point of view for regularizing deep neural networks by using the norm of a reproducing kernel Hilbert space (RKHS).
On the Inductive Bias of Neural Tangent Kernels
State-of-the-art neural networks are heavily over-parameterized, making the optimization algorithm a crucial ingredient for learning predictive models with good generalization properties.
Counterfactual Learning of Stochastic Policies with Continuous Actions: from Models to Offline Evaluation
Counterfactual reasoning from logged data has become increasingly important for many applications such as web advertising or healthcare.
Convergence and Stability of Graph Convolutional Networks on Large Random Graphs
We study properties of Graph Convolutional Networks (GCNs) by analyzing their behavior on standard models of random graphs, where nodes are represented by random latent variables and edges are drawn according to a similarity kernel.
Deep Equals Shallow for ReLU Networks in Kernel Regimes
Deep networks are often considered to be more expressive than shallow ones in terms of approximation.
Approximation and Learning with Deep Convolutional Models: a Kernel Perspective
The empirical success of deep convolutional networks on tasks involving high-dimensional data such as images or audio suggests that they can efficiently approximate certain functions that are well-suited for such tasks.
On Energy-Based Models with Overparametrized Shallow Neural Networks
Energy-based models (EBMs) are a simple yet powerful framework for generative modeling.
On the Universality of Graph Neural Networks on Large Random Graphs
In the large graph limit, GNNs are known to converge to certain "continuous" models known as c-GNNs, which directly enables a study of their approximation power on random graph models.
On the Sample Complexity of Learning under Invariance and Geometric Stability
Many supervised learning problems involve high-dimensional data such as images, text, or graphs.
Dual Training of Energy-Based Models with Overparametrized Shallow Neural Networks
In the feature-learning regime, this dual formulation justifies using a two time-scale gradient ascent-descent (GDA) training algorithm in which one updates concurrently the particles in the sample space and the neurons in the parameter space of the energy.
On the Sample Complexity of Learning under Geometric Stability
Many supervised learning problems involve high-dimensional data such as images, text, or graphs.
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning
Large-scale machine learning systems often involve data distributed across a collection of users.
Efficient Kernel UCB for Contextual Bandits
While standard methods require a O(CT^3) complexity where T is the horizon and the constant C is related to optimizing the UCB rule, we propose an efficient contextual algorithm for large-scale problems.
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes
To better understand these factors, we provide a precise study of the adversarial robustness in different scenarios, from initialization to the end of training in different regimes, as well as intermediate scenarios, where initialization still plays a role due to "lazy" training.
When does return-conditioned supervised learning work for offline reinforcement learning?
Several recent works have proposed a class of algorithms for the offline reinforcement learning (RL) problem that we will refer to as return-conditioned supervised learning (RCSL).
Learning Single-Index Models with Shallow Neural Networks
Single-index models are a class of functions given by an unknown univariate ``link'' function applied to an unknown one-dimensional projection of the input.
On minimal variations for unsupervised representation learning
Unsupervised representation learning aims at describing raw data efficiently to solve various downstream tasks.
The SSL Interplay: Augmentations, Inductive Bias, and Generalization
Self-supervised learning (SSL) has emerged as a powerful framework to learn representations from raw data without supervision.
AstroCLIP: Cross-Modal Pre-Training for Astronomical Foundation Models
We present AstroCLIP, a strategy to facilitate the construction of astronomical foundation models that bridge the gap between diverse observational modalities.
Multiple Physics Pretraining for Physical Surrogate Models
We introduce multiple physics pretraining (MPP), an autoregressive task-agnostic pretraining approach for physical surrogate modeling.
xVal: A Continuous Number Encoding for Large Language Models
Large Language Models have not yet been broadly adapted for the analysis of scientific datasets due in part to the unique difficulties of tokenizing numbers.
Scaling Laws for Associative Memories
Learning arguably involves the discovery and memorization of abstract rules.
On Learning Gaussian Multi-index Models with Gradient Flow
We study gradient flow on the multi-index regression problem for high-dimensional Gaussian data.
Learning Associative Memories with Gradient Descent
This work focuses on the training dynamics of one associative memory module storing outer products of token embeddings.
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
We show that the heavy-tailed class imbalance found in language modeling tasks leads to difficulties in the optimization dynamics.
Level Set Teleportation: An Optimization Perspective
We study level set teleportation, an optimization sub-routine which seeks to accelerate gradient methods by maximizing the gradient norm on a level-set of the objective function.
