Search Results for author:
Found 47 papers, 6 papers with code
Gradient Descent Converges to Minimizers
We show that gradient descent converges to a local minimizer, almost surely with random initialization.
Best-of-K Bandits
This paper studies the Best-of-K Bandit game: At each time the player chooses a subset S among all N-choose-K possible options and observes reward max(X(i) : i in S) where X is a random vector drawn from a joint distribution.
The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime
Moreover, our lower bounds zero-in on the number of times each \emph{individual} arm needs to be pulled, uncovering new phenomena which are drowned out in the aggregate sample complexity.
On the Gap Between Strict-Saddles and True Convexity: An Omega(log d) Lower Bound for Eigenvector Approximation
We prove a \emph{query complexity} lower bound on rank-one principal component analysis (PCA).
First-order Methods Almost Always Avoid Saddle Points
We establish that first-order methods avoid saddle points for almost all initializations.
Approximate Ranking from Pairwise Comparisons
Accordingly, we study the problem of finding approximate rankings from pairwise comparisons.
Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification
We prove that the ordinary least-squares (OLS) estimator attains nearly minimax optimal performance for the identification of linear dynamical systems from a single observed trajectory.
Delayed Impact of Fair Machine Learning
Fairness in machine learning has predominantly been studied in static classification settings without concern for how decisions change the underlying population over time.
Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law
We show that for every $\mathtt{gap} \in (0, 1/2]$, there exists a distribution over matrices $\mathbf{M}$ for which 1) $\mathrm{gap}_r(\mathbf{M}) = \Omega(\mathtt{gap})$ (where $\mathrm{gap}_r(\mathbf{M})$ is the normalized gap between the $r$ and $r+1$-st largest-magnitude eigenvector of $\mathbf{M}$), and 2) any algorithm $\mathsf{Alg}$ which takes fewer than $\mathrm{const} \times \frac{r \log d}{\sqrt{\mathtt{gap}}}$ queries fails (with overwhelming probability) to identity a matrix $\widehat{\mathsf{V}} \in \mathbb{R}^{d \times r}$ with orthonormal columns for which $\langle \widehat{\mathsf{V}}, \mathbf{M} \widehat{\mathsf{V}}\rangle \ge (1 - \mathrm{const} \times \mathtt{gap})\sum_{i=1}^r \lambda_i(\mathbf{M})$.
On the Randomized Complexity of Minimizing a Convex Quadratic Function
Minimizing a convex, quadratic objective of the form $f_{\mathbf{A},\mathbf{b}}(x) := \frac{1}{2}x^\top \mathbf{A} x - \langle \mathbf{b}, x \rangle$ for $\mathbf{A} \succ 0 $ is a fundamental problem in machine learning and optimization.
Adaptive Sampling for Convex Regression
In this paper, we introduce the first principled adaptive-sampling procedure for learning a convex function in the $L_\infty$ norm, a problem that arises often in the behavioral and social sciences.
The implicit fairness criterion of unconstrained learning
We show that under reasonable conditions, the deviation from satisfying group calibration is upper bounded by the excess risk of the learned score relative to the Bayes optimal score function.
A Successive-Elimination Approach to Adaptive Robotic Sensing
We study an adaptive source seeking problem, in which a mobile robot must identify the strongest emitter(s) of a signal in an environment with background emissions.
Learning Linear Dynamical Systems with Semi-Parametric Least Squares
We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning.
Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPs
This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs.
The gradient complexity of linear regression
We investigate the computational complexity of several basic linear algebra primitives, including largest eigenvector computation and linear regression, in the computational model that allows access to the data via a matrix-vector product oracle.
Corruption-robust exploration in episodic reinforcement learning
We initiate the study of multi-stage episodic reinforcement learning under adversarial corruptions in both the rewards and the transition probabilities of the underlying system extending recent results for the special case of stochastic bandits.
Improper Learning for Non-Stochastic Control
We consider the problem of controlling a possibly unknown linear dynamical system with adversarial perturbations, adversarially chosen convex loss functions, and partially observed states, known as non-stochastic control.
Naive Exploration is Optimal for Online LQR
Our upper bound is attained by a simple variant of $\textit{{certainty equivalent control}}$, where the learner selects control inputs according to the optimal controller for their estimate of the system while injecting exploratory random noise.
Reward-Free Exploration for Reinforcement Learning
We give an efficient algorithm that conducts $\tilde{\mathcal{O}}(S^2A\mathrm{poly}(H)/\epsilon^2)$ episodes of exploration and returns $\epsilon$-suboptimal policies for an arbitrary number of reward functions.
Logarithmic Regret for Adversarial Online Control
We introduce a new algorithm for online linear-quadratic control in a known system subject to adversarial disturbances.
Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning
Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies.
Constrained episodic reinforcement learning in concave-convex and knapsack settings
We propose an algorithm for tabular episodic reinforcement learning with constraints.
Making Non-Stochastic Control (Almost) as Easy as Stochastic
Recent literature has made much progress in understanding \emph{online LQR}: a modern learning-theoretic take on the classical control problem in which a learner attempts to optimally control an unknown linear dynamical system with fully observed state, perturbed by i. i. d.
Learning the Linear Quadratic Regulator from Nonlinear Observations
We introduce a new algorithm, RichID, which learns a near-optimal policy for the RichLQR with sample complexity scaling only with the dimension of the latent state space and the capacity of the decoder function class.
Task-Optimal Exploration in Linear Dynamical Systems
Along the way, we establish that certainty equivalence decision making is instance- and task-optimal, and obtain the first algorithm for the linear quadratic regulator problem which is instance-optimal.
Exploration and Incentives in Reinforcement Learning
How do you incentivize self-interested agents to $\textit{explore}$ when they prefer to $\textit{exploit}$?
Towards a Dimension-Free Understanding of Adaptive Linear Control
We study the problem of adaptive control of the linear quadratic regulator for systems in very high, or even infinite dimension.
On the Stability of Nonlinear Receding Horizon Control: A Geometric Perspective
Crucially, this guarantee requires that state costs applied to the planning problems are in a certain sense `compatible' with the global geometry of the system, and a simple counter-example demonstrates the necessity of this condition.
Bayesian decision-making under misspecified priors with applications to meta-learning
We prove that the expected reward accrued by Thompson sampling (TS) with a misspecified prior differs by at most $\tilde{\mathcal{O}}(H^2 \epsilon)$ from TS with a well specified prior, where $\epsilon$ is the total-variation distance between priors and $H$ is the learning horizon.
Beyond No Regret: Instance-Dependent PAC Reinforcement Learning
We show this is not possible -- there exists a fundamental tradeoff between achieving low regret and identifying an $\epsilon$-optimal policy at the instance-optimal rate.
Stabilizing Dynamical Systems via Policy Gradient Methods
Stabilizing an unknown control system is one of the most fundamental problems in control systems engineering.
First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approach
Obtaining first-order regret bounds -- regret bounds scaling not as the worst-case but with some measure of the performance of the optimal policy on a given instance -- is a core question in sequential decision-making.
Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes
We first develop a computationally efficient algorithm for reward-free RL in a $d$-dimensional linear MDP with sample complexity scaling as $\widetilde{\mathcal{O}}(d^2 H^5/\epsilon^2)$.
Do Differentiable Simulators Give Better Policy Gradients?
Differentiable simulators promise faster computation time for reinforcement learning by replacing zeroth-order gradient estimates of a stochastic objective with an estimate based on first-order gradients.
Online Control of Unknown Time-Varying Dynamical Systems
On the positive side, we give an efficient algorithm that attains a sublinear regret bound against the class of Disturbance Response policies up to the aforementioned system variability term.
Globally Convergent Policy Search over Dynamic Filters for Output Estimation
In this paper, we provide a new perspective on this challenging problem based on the notion of $\textit{informativity}$, which intuitively requires that all components of a filter's internal state are representative of the true state of the underlying dynamical system.
Efficient and Near-Optimal Smoothed Online Learning for Generalized Linear Functions
Due to the drastic gap in complexity between sequential and batch statistical learning, recent work has studied a smoothed sequential learning setting, where Nature is constrained to select contexts with density bounded by 1/{\sigma} with respect to a known measure {\mu}.
Smoothed Online Learning for Prediction in Piecewise Affine Systems
The problem of piecewise affine (PWA) regression and planning is of foundational importance to the study of online learning, control, and robotics, where it provides a theoretically and empirically tractable setting to study systems undergoing sharp changes in the dynamics.
Oracle-Efficient Smoothed Online Learning for Piecewise Continuous Decision Making
Smoothed online learning has emerged as a popular framework to mitigate the substantial loss in statistical and computational complexity that arises when one moves from classical to adversarial learning.
Statistical Learning under Heterogeneous Distribution Shift
We show that, when the class $F$ is "simpler" than $G$ (measured, e. g., in terms of its metric entropy), our predictor is more resilient to heterogeneous covariate shifts} in which the shift in $\mathbf{x}$ is much greater than that in $\mathbf{y}$.
Learning to Extrapolate: A Transductive Approach
Machine learning systems, especially with overparameterized deep neural networks, can generalize to novel test instances drawn from the same distribution as the training data.
The Power of Learned Locally Linear Models for Nonlinear Policy Optimization
A common pipeline in learning-based control is to iteratively estimate a model of system dynamics, and apply a trajectory optimization algorithm - e. g.~$\mathtt{iLQR}$ - on the learned model to minimize a target cost.
Tackling Combinatorial Distribution Shift: A Matrix Completion Perspective
Focusing on the special case where the labels are given by bilinear embeddings into a Hilbert space $H$: $\mathbb{E}[z \mid x, y ]=\langle f_{\star}(x), g_{\star}(y)\rangle_{{H}}$, we aim to extrapolate to a test distribution domain that is $not$ covered in training, i. e., achieving bilinear combinatorial extrapolation.
Robot Fleet Learning via Policy Merging
We show that FLEET-MERGE consolidates the behavior of policies trained on 50 tasks in the Meta-World environment, with good performance on nearly all training tasks at test time.
Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression
This work studies training instabilities of behavior cloning with deep neural networks.
Logarithmic Regret for Online Control with Adversarial Noise
We consider the problem of online control in a known linear dynamical system subject to adversarial noise.
