Search Results for author:
Found 35 papers, 10 papers with code
The Tradeoffs of Large Scale Learning
This contribution develops a theoretical framework that takes into account the effect of approximate optimization on learning algorithms.
AdaGAN: Boosting Generative Models
Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are an effective method for training generative models of complex data such as natural images.
From optimal transport to generative modeling: the VEGAN cookbook
We study unsupervised generative modeling in terms of the optimal transport (OT) problem between true (but unknown) data distribution $P_X$ and the latent variable model distribution $P_G$.
Better Text Understanding Through Image-To-Text Transfer
Generic text embeddings are successfully used in a variety of tasks.
Approximation and Convergence Properties of Generative Adversarial Learning
In this paper, we address these questions in a broad and unified setting by defining a notion of adversarial divergences that includes a number of recently proposed objective functions.
Toward Optimal Run Racing: Application to Deep Learning Calibration
This paper aims at one-shot learning of deep neural nets, where a highly parallel setting is considered to address the algorithm calibration problem - selecting the best neural architecture and learning hyper-parameter values depending on the dataset at hand.
Critical Hyper-Parameters: No Random, No Cry
The selection of hyper-parameters is critical in Deep Learning.
Wasserstein Auto-Encoders
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution.
Are GANs Created Equal? A Large-Scale Study
Generative adversarial networks (GAN) are a powerful subclass of generative models.
Online Hyper-Parameter Optimization
We propose an efficient online hyperparameter optimization method which uses a joint dynamical system to evaluate the gradient with respect to the hyperparameters.
Gradient Descent Quantizes ReLU Network Features
Deep neural networks are often trained in the over-parametrized regime (i. e. with far more parameters than training examples), and understanding why the training converges to solutions that generalize remains an open problem.
Assessing Generative Models via Precision and Recall
Recent advances in generative modeling have led to an increased interest in the study of statistical divergences as means of model comparison.
Synthetic Data Generators: Sequential and Private
We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient).
The Optimal Approximation Factor in Density Estimation
We complement and extend this result by showing that: (i) the factor 3 can not be improved if one restricts the algorithm to output a density from $\mathcal{Q}$, and (ii) if one allows the algorithm to output arbitrary densities (e. g.\ a mixture of densities from $\mathcal{Q}$), then the approximation factor can be reduced to 2, which is optimal.
Precision-Recall Curves Using Information Divergence Frontiers
Despite the tremendous progress in the estimation of generative models, the development of tools for diagnosing their failures and assessing their performance has advanced at a much slower pace.
Practical and Consistent Estimation of f-Divergences
The estimation of an f-divergence between two probability distributions based on samples is a fundamental problem in statistics and machine learning.
When can unlabeled data improve the learning rate?
In semi-supervised classification, one is given access both to labeled and unlabeled data.
Google Research Football: A Novel Reinforcement Learning Environment
Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly tested in a safe and reproducible manner.
The Visual Task Adaptation Benchmark
Representation learning promises to unlock deep learning for the long tail of vision tasks without expansive labelled datasets.
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
And, how close are we to general visual representations?
Ranked #10 on
Image Classification
on VTAB-1k
(using extra training data)
Sharper bounds for uniformly stable algorithms
In a series of recent breakthrough papers by Feldman and Vondrak (2018, 2019), it was shown that the best known high probability upper bounds for uniformly stable learning algorithms due to Bousquet and Elisseef (2002) are sub-optimal in some natural regimes.
Fast classification rates without standard margin assumptions
First, we consider classification with a reject option, namely Chow's reject option model, and show that by slightly lowering the impact of hard instances, a learning rate of order $O\left(\frac{d}{n}\log \frac{n}{d}\right)$ is always achievable in the agnostic setting by a specific learning algorithm.
Measuring Compositional Generalization: A Comprehensive Method on Realistic Data
We present a large and realistic natural language question answering dataset that is constructed according to this method, and we use it to analyze the compositional generalization ability of three machine learning architectures.
Ranked #5 on
Semantic Parsing
on CFQ
Predicting Neural Network Accuracy from Weights
Furthermore, the predictors are able to rank networks trained on different, unobserved datasets and with different architectures.
Proper Learning, Helly Number, and an Optimal SVM Bound
It has been recently shown by Hanneke (2016) that the optimal sample complexity of PAC learning for any VC class C is achieved by a particular improper learning algorithm, which outputs a specific majority-vote of hypotheses in C. This leaves the question of when this bound can be achieved by proper learning algorithms, which are restricted to always output a hypothesis from C. In this paper we aim to characterize the classes for which the optimal sample complexity can be achieved by a proper learning algorithm.
What Do Neural Networks Learn When Trained With Random Labels?
We show how this alignment produces a positive transfer: networks pre-trained with random labels train faster downstream compared to training from scratch even after accounting for simple effects, such as weight scaling.
Synthetic Data Generators -- Sequential and Private
We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient).
Statistically Near-Optimal Hypothesis Selection
We derive an optimal $2$-approximation learning strategy for the Hypothesis Selection problem, outputting $q$ such that $\mathsf{TV}(p, q) \leq2 \cdot opt + \eps$, with a (nearly) optimal sample complexity of~$\tilde O(\log n/\epsilon^2)$.
Monotone Learning
Our transformation readily implies monotone learners in a variety of contexts: for example it extends Pestov's result to classification tasks with an arbitrary number of labels.
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Chain-of-thought prompting has demonstrated remarkable performance on various natural language reasoning tasks.
Ranked #96 on
Arithmetic Reasoning
on GSM8K
Fine-Grained Distribution-Dependent Learning Curves
We solve this problem in a principled manner, by introducing a combinatorial dimension called VCL that characterizes the best $d'$ for which $d'/n$ is a strong minimax lower bound.
Compositional Semantic Parsing with Large Language Models
Humans can reason compositionally when presented with new tasks.
Ranked #1 on
Semantic Parsing
on CFQ
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima
We consider Sharpness-Aware Minimization (SAM), a gradient-based optimization method for deep networks that has exhibited performance improvements on image and language prediction problems.
Differentially-Private Bayes Consistency
We construct a universally Bayes consistent learning rule that satisfies differential privacy (DP).
