Search Results for author:
Found 54 papers, 17 papers with code
Learning from Logged Implicit Exploration Data
We provide a sound and consistent foundation for the use of \emph{nonrandom} exploration data in "contextual bandit" or "partially labeled" settings where only the value of a chosen action is learned.
When are Overcomplete Topic Models Identifiable? Uniqueness of Tensor Tucker Decompositions with Structured Sparsity
This set of higher-order expansion conditions allow for overcomplete models, and require the existence of a perfect matching from latent topics to higher order observed words.
Minimal Realization Problems for Hidden Markov Models
Consider a stationary discrete random process with alphabet size d, which is assumed to be the output process of an unknown stationary Hidden Markov Model (HMM).
A Linear Dynamical System Model for Text
Finally, the Kalman filter updates can be seen as a linear recurrent neural network.
Convergence Rates of Active Learning for Maximum Likelihood Estimation
Provided certain conditions hold on the model class, we provide a two-stage active learning algorithm for this problem.
Learning Features of Music from Scratch
This paper introduces a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research.
Ranked #6 on
Music Transcription
on MusicNet
Prediction with a Short Memory
For a Hidden Markov Model with $n$ hidden states, $I$ is bounded by $\log n$, a quantity that does not depend on the mixing time, and we show that the trivial prediction algorithm based on the empirical frequencies of length $O(\log n/\epsilon)$ windows of observations achieves this error, provided the length of the sequence is $d^{\Omega(\log n/\epsilon)}$, where $d$ is the size of the observation alphabet.
Towards Generalization and Simplicity in Continuous Control
This work shows that policies with simple linear and RBF parameterizations can be trained to solve a variety of continuous control tasks, including the OpenAI gym benchmarks.
Learning Overcomplete HMMs
On the other hand, we show that learning is impossible given only a polynomial number of samples for HMMs with a small output alphabet and whose transition matrices are random regular graphs with large degree.
Leverage Score Sampling for Faster Accelerated Regression and ERM
Given a matrix $\mathbf{A}\in\mathbb{R}^{n\times d}$ and a vector $b \in\mathbb{R}^{d}$, we show how to compute an $\epsilon$-approximate solution to the regression problem $ \min_{x\in\mathbb{R}^{d}}\frac{1}{2} \|\mathbf{A} x - b\|_{2}^{2} $ in time $ \tilde{O} ((n+\sqrt{d\cdot\kappa_{\text{sum}}})\cdot s\cdot\log\epsilon^{-1}) $ where $\kappa_{\text{sum}}=\mathrm{tr}\left(\mathbf{A}^{\top}\mathbf{A}\right)/\lambda_{\min}(\mathbf{A}^{T}\mathbf{A})$ and $s$ is the maximum number of non-zero entries in a row of $\mathbf{A}$.
Variance Reduction Methods for Sublinear Reinforcement Learning
There is a technical issue in the analysis that is not easily fixable.
Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines
To mitigate this issue, we derive a bias-free action-dependent baseline for variance reduction which fully exploits the structural form of the stochastic policy itself and does not make any additional assumptions about the MDP.
Stochastic subgradient method converges on tame functions
This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity?
Provably Correct Automatic Subdifferentiation for Qualified Programs
The Cheap Gradient Principle (Griewank 2008) --- the computational cost of computing the gradient of a scalar-valued function is nearly the same (often within a factor of $5$) as that of simply computing the function itself --- is of central importance in optimization; it allows us to quickly obtain (high dimensional) gradients of scalar loss functions which are subsequently used in black box gradient-based optimization procedures.
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
We study how local trajectory optimization can cope with approximation errors in the value function, and can stabilize and accelerate value function learning.
Online Meta-Learning
Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adaptation on a new task, but typically assumes the set of tasks are available together as a batch.
Model-Based Reinforcement Learning with a Generative Model is Minimax Optimal
In this work, we study the effectiveness of the most natural plug-in approach to model-based planning: we build the maximum likelihood estimate of the transition model in the MDP from observations and then find an optimal policy in this empirical MDP.
Model-based Reinforcement Learning
reinforcement-learning
+1
Meta-Learning with Implicit Gradients
By drawing upon implicit differentiation, we develop the implicit MAML algorithm, which depends only on the solution to the inner level optimization and not the path taken by the inner loop optimizer.
Soft Threshold Weight Reparameterization for Learnable Sparsity
Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget.
Meta-learning for mixed linear regression
In modern supervised learning, there are a large number of tasks, but many of them are associated with only a small amount of labeled data.
Provable Representation Learning for Imitation Learning via Bi-level Optimization
We formulate representation learning as a bi-level optimization problem where the "outer" optimization tries to learn the joint representation and the "inner" optimization encodes the imitation learning setup and tries to learn task-specific parameters.
The Implicit and Explicit Regularization Effects of Dropout
This implicit regularization effect is analogous to the effect of stochasticity in small mini-batch stochastic gradient descent.
Optimal Regularization Can Mitigate Double Descent
Recent empirical and theoretical studies have shown that many learning algorithms -- from linear regression to neural networks -- can have test performance that is non-monotonic in quantities such the sample size and model size.
Robust Meta-learning for Mixed Linear Regression with Small Batches
Together, this approach is robust against outliers and achieves a graceful statistical trade-off; the lack of $\Omega(k^{1/2})$-size tasks can be compensated for with smaller tasks, which can now be as small as $O(\log k)$.
FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs
In order to deal with the curse of dimensionality in reinforcement learning (RL), it is common practice to make parametric assumptions where values or policies are functions of some low dimensional feature space.
Information Theoretic Regret Bounds for Online Nonlinear Control
This work studies the problem of sequential control in an unknown, nonlinear dynamical system, where we model the underlying system dynamics as an unknown function in a known Reproducing Kernel Hilbert Space.
PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learning
Direct policy gradient methods for reinforcement learning are a successful approach for a variety of reasons: they are model free, they directly optimize the performance metric of interest, and they allow for richly parameterized policies.
How Important is the Train-Validation Split in Meta-Learning?
A common practice in meta-learning is to perform a train-validation split (\emph{train-val method}) where the prior adapts to the task on one split of the data, and the resulting predictor is evaluated on another split.
Is Long Horizon RL More Difficult Than Short Horizon RL?
In a COLT 2018 open problem, Jiang and Agarwal conjectured that, for tabular, episodic reinforcement learning problems, there exists a sample complexity lower bound which exhibits a polynomial dependence on the horizon --- a conjecture which is consistent with all known sample complexity upper bounds.
Robust and Differentially Private Mean Estimation
In statistical learning and analysis from shared data, which is increasingly widely adopted in platforms such as federated learning and meta-learning, there are two major concerns: privacy and robustness.
LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes
We further quantitatively measure the quality of our codes by applying it to the efficient image retrieval as well as out-of-distribution (OOD) detection problems.
Gone Fishing: Neural Active Learning with Fisher Embeddings
There is an increasing need for effective active learning algorithms that are compatible with deep neural networks.
Koopman Spectrum Nonlinear Regulator and Provably Efficient Online Learning
In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics.
Sparsity in Partially Controllable Linear Systems
However, in practice, we often encounter systems in which a large set of state variables evolve exogenously and independently of the control inputs; such systems are only partially controllable.
Inductive Biases and Variable Creation in Self-Attention Mechanisms
Self-attention, an architectural motif designed to model long-range interactions in sequential data, has driven numerous recent breakthroughs in natural language processing and beyond.
Anti-Concentrated Confidence Bonuses for Scalable Exploration
Intrinsic rewards play a central role in handling the exploration-exploitation trade-off when designing sequential decision-making algorithms, in both foundational theory and state-of-the-art deep reinforcement learning.
Multi-Stage Episodic Control for Strategic Exploration in Text Games
Text adventure games present unique challenges to reinforcement learning methods due to their combinatorially large action spaces and sparse rewards.
Understanding Contrastive Learning Requires Incorporating Inductive Biases
Contrastive learning is a popular form of self-supervised learning that encourages augmentations (views) of the same input to have more similar representations compared to augmentations of different inputs.
A Complete Characterization of Linear Estimators for Offline Policy Evaluation
Offline policy evaluation is a fundamental statistical problem in reinforcement learning that involves estimating the value function of some decision-making policy given data collected by a potentially different policy.
Matryoshka Representation Learning
The flexibility within the learned Matryoshka Representations offer: (a) up to 14x smaller embedding size for ImageNet-1K classification at the same level of accuracy; (b) up to 14x real-world speed-ups for large-scale retrieval on ImageNet-1K and 4K; and (c) up to 2% accuracy improvements for long-tail few-shot classification, all while being as robust as the original representations.
Ranked #25 on
Image Classification
on ObjectNet
(using extra training data)
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit
There is mounting evidence of emergent phenomena in the capabilities of deep learning methods as we scale up datasets, model sizes, and training times.
Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms
For example, on parity problems, the NN learns as well as Gaussian elimination, an efficient algorithm that can be succinctly described.
On Provable Copyright Protection for Generative Models
There is a growing concern that learned conditional generative models may output samples that are substantially similar to some copyrighted data $C$ that was in their training set.
Modified Gauss-Newton Algorithms under Noise
Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing.
AdANNS: A Framework for Adaptive Semantic Search
Finally, we demonstrate that AdANNS can enable inference-time adaptivity for compute-aware search on ANNS indices built non-adaptively on matryoshka representations.
Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
The success of SGD in deep learning has been ascribed by prior works to the implicit bias induced by high learning rate or small batch size ("SGD noise").
Scaling Laws for Imitation Learning in Single-Agent Games
Inspired by recent work in Natural Language Processing (NLP) where "scaling up" has resulted in increasingly more capable LLMs, we investigate whether carefully scaling up model and data size can bring similar improvements in the imitation learning setting for single-agent games.
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck
Finally, we show that the synthetic sparse parity task can be useful as a proxy for real problems requiring axis-aligned feature learning.
MatFormer: Nested Transformer for Elastic Inference
Furthermore, we observe that smaller encoders extracted from a universal MatFormer-based ViT (MatViT) encoder preserve the metric-space structure for adaptive large-scale retrieval.
Learning an Inventory Control Policy with General Inventory Arrival Dynamics
In this paper we address the problem of learning and backtesting inventory control policies in the presence of general arrival dynamics -- which we term as a quantity-over-time arrivals model (QOT).
Feature emergence via margin maximization: case studies in algebraic tasks
Understanding the internal representations learned by neural networks is a cornerstone challenge in the science of machine learning.
A Study on the Calibration of In-context Learning
Accurate uncertainty quantification is crucial for the safe deployment of machine learning models, and prior research has demonstrated improvements in the calibration of modern language models (LMs).
Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
The idea is to learn a simple linear function on a model's embedding space that can be used to reweight candidate completions.
Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems
Retrieval-Augmented Generation (RAG) improves pre-trained models by incorporating external knowledge at test time to enable customized adaptation.
