Search Results for author: Benjamin Recht

Found 80 papers, 30 papers with code

Exact Matrix Completion via Convex Optimization

no code implementations29 May 2008 Emmanuel J. Candes, Benjamin Recht

We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries.

Information Theory Information Theory

Online Identification and Tracking of Subspaces from Highly Incomplete Information

1 code implementation21 Jun 2010 Laura Balzano, Robert Nowak, Benjamin Recht

GROUSE performs exceptionally well in practice both in tracking subspaces and as an online algorithm for matrix completion.

Matrix Completion

HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

5 code implementations28 Jun 2011 Feng Niu, Benjamin Recht, Christopher Re, Stephen J. Wright

Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks.

Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

no code implementations NeurIPS 2011 Benjamin Recht, Christopher Re, Stephen Wright, Feng Niu

Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks.

Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences

3 code implementations19 Feb 2012 Benjamin Recht, Christopher Re

We detail the consequences of this inequality for stochastic gradient descent and the randomized Kaczmarz algorithm for solving linear systems.

Atomic norm denoising with applications to line spectral estimation

1 code implementation3 Apr 2012 Badri Narayan Bhaskar, Gongguo Tang, Benjamin Recht

Motivated by recent work on atomic norms in inverse problems, we propose a new approach to line spectral estimation that provides theoretical guarantees for the mean-squared-error (MSE) performance in the presence of noise and without knowledge of the model order.

Information Theory Information Theory

Factoring nonnegative matrices with linear programs

1 code implementation NeurIPS 2012 Victor Bittorf, Benjamin Recht, Christopher Re, Joel A. Tropp

The constraints are chosen to ensure that the matrix C selects features; these features can then be used to find a low-rank NMF of X.

Blind Deconvolution using Convex Programming

1 code implementation21 Nov 2012 Ali Ahmed, Benjamin Recht, Justin Romberg

That is, we show that if $\boldsymbol{x}$ is drawn from a random subspace of dimension $N$, and $\boldsymbol{w}$ is a vector in a subspace of dimension $K$ whose basis vectors are "spread out" in the frequency domain, then nuclear norm minimization recovers $\boldsymbol{w}\boldsymbol{x}^*$ without error.

Information Theory Information Theory

Compressive classification and the rare eclipse problem

no code implementations11 Apr 2014 Afonso S. Bandeira, Dustin G. Mixon, Benjamin Recht

This paper addresses the fundamental question of when convex sets remain disjoint after random projection.

Classification General Classification

The Randomized Causation Coefficient

no code implementations15 Sep 2014 David Lopez-Paz, Krikamol Muandet, Benjamin Recht

We are interested in learning causal relationships between pairs of random variables, purely from observational data.

Causal Inference Feature Engineering

A General Analysis of the Convergence of ADMM

no code implementations6 Feb 2015 Robert Nishihara, Laurent Lessard, Benjamin Recht, Andrew Packard, Michael. I. Jordan

We provide a new proof of the linear convergence of the alternating direction method of multipliers (ADMM) when one of the objective terms is strongly convex.

Optimization and Control Numerical Analysis

Isometric sketching of any set via the Restricted Isometry Property

no code implementations11 Jun 2015 Samet Oymak, Benjamin Recht, Mahdi Soltanolkotabi

In this paper we show that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices.

Dimensionality Reduction

Sharp Time--Data Tradeoffs for Linear Inverse Problems

no code implementations16 Jul 2015 Samet Oymak, Benjamin Recht, Mahdi Soltanolkotabi

We sharply characterize the convergence rate associated with a wide variety of random measurement ensembles in terms of the number of measurements and structural complexity of the signal with respect to the chosen penalty function.

Parallel Correlation Clustering on Big Graphs

no code implementations NeurIPS 2015 Xinghao Pan, Dimitris Papailiopoulos, Samet Oymak, Benjamin Recht, Kannan Ramchandran, Michael. I. Jordan

We present C4 and ClusterWild!, two algorithms for parallel correlation clustering that run in a polylogarithmic number of rounds and achieve nearly linear speedups, provably.

Clustering

Perturbed Iterate Analysis for Asynchronous Stochastic Optimization

no code implementations24 Jul 2015 Horia Mania, Xinghao Pan, Dimitris Papailiopoulos, Benjamin Recht, Kannan Ramchandran, Michael. I. Jordan

We demonstrate experimentally on a 16-core machine that the sparse and parallel version of SVRG is in some cases more than four orders of magnitude faster than the standard SVRG algorithm.

Stochastic Optimization

Train faster, generalize better: Stability of stochastic gradient descent

no code implementations3 Sep 2015 Moritz Hardt, Benjamin Recht, Yoram Singer

In the non-convex case, we give a new interpretation of common practices in neural networks, and formally show that popular techniques for training large deep models are indeed stability-promoting.

Gradient Descent Converges to Minimizers

no code implementations16 Feb 2016 Jason D. Lee, Max Simchowitz, Michael. I. Jordan, Benjamin Recht

We show that gradient descent converges to a local minimizer, almost surely with random initialization.

Large Scale Kernel Learning using Block Coordinate Descent

no code implementations17 Feb 2016 Stephen Tu, Rebecca Roelofs, Shivaram Venkataraman, Benjamin Recht

We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points.

Classification General Classification +1

Best-of-K Bandits

no code implementations9 Mar 2016 Max Simchowitz, Kevin Jamieson, Benjamin Recht

This paper studies the Best-of-K Bandit game: At each time the player chooses a subset S among all N-choose-K possible options and observes reward max(X(i) : i in S) where X is a random vector drawn from a joint distribution.

On kernel methods for covariates that are rankings

no code implementations25 Mar 2016 Horia Mania, Aaditya Ramdas, Martin J. Wainwright, Michael. I. Jordan, Benjamin Recht

This paper studies the use of reproducing kernel Hilbert space methods for learning from permutation-valued features.

regression

Gradient Descent Learns Linear Dynamical Systems

no code implementations16 Sep 2016 Moritz Hardt, Tengyu Ma, Benjamin Recht

We prove that stochastic gradient descent efficiently converges to the global optimizer of the maximum likelihood objective of an unknown linear time-invariant dynamical system from a sequence of noisy observations generated by the system.

Saturating Splines and Feature Selection

no code implementations21 Sep 2016 Nicholas Boyd, Trevor Hastie, Stephen Boyd, Benjamin Recht, Michael Jordan

We extend the adaptive regression spline model by incorporating saturation, the natural requirement that a function extend as a constant outside a certain range.

Additive models feature selection

KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics

no code implementations29 Oct 2016 Evan R. Sparks, Shivaram Venkataraman, Tomer Kaftan, Michael J. Franklin, Benjamin Recht

Modern advanced analytics applications make use of machine learning techniques and contain multiple steps of domain-specific and general-purpose processing with high resource requirements.

BIG-bench Machine Learning General Classification +1

Understanding deep learning requires rethinking generalization

7 code implementations10 Nov 2016 Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, Oriol Vinyals

Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.

Image Classification

The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime

no code implementations16 Feb 2017 Max Simchowitz, Kevin Jamieson, Benjamin Recht

Moreover, our lower bounds zero-in on the number of times each \emph{individual} arm needs to be pulled, uncovering new phenomena which are drowned out in the aggregate sample complexity.

The Marginal Value of Adaptive Gradient Methods in Machine Learning

3 code implementations NeurIPS 2017 Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nathan Srebro, Benjamin Recht

Adaptive optimization methods, which perform local optimization with a metric constructed from the history of iterates, are becoming increasingly popular for training deep neural networks.

BIG-bench Machine Learning Binary Classification

Non-Asymptotic Analysis of Robust Control from Coarse-Grained Identification

no code implementations15 Jul 2017 Stephen Tu, Ross Boczar, Andrew Packard, Benjamin Recht

We derive bounds on the number of noisy input/output samples from a stable linear time-invariant system that are sufficient to guarantee that the corresponding finite impulse response approximation is close to the true system in the $\mathcal{H}_\infty$-norm.

Flare Prediction Using Photospheric and Coronal Image Data

no code implementations3 Aug 2017 Eric Jonas, Monica G. Bobra, Vaishaal Shankar, J. Todd Hoeksema, Benjamin Recht

This is the first attempt to predict solar flares using photospheric vector magnetic field data as well as multiple wavelengths of image data from the chromosphere, transition region, and corona.

On the Sample Complexity of the Linear Quadratic Regulator

no code implementations4 Oct 2017 Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, Stephen Tu

This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown.

First-order Methods Almost Always Avoid Saddle Points

no code implementations20 Oct 2017 Jason D. Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael. I. Jordan, Benjamin Recht

We establish that first-order methods avoid saddle points for almost all initializations.

Massively Parallel Hyperparameter Tuning

no code implementations ICLR 2018 Lisha Li, Kevin Jamieson, Afshin Rostamizadeh, Katya Gonina, Moritz Hardt, Benjamin Recht, Ameet Talwalkar

Modern machine learning models are characterized by large hyperparameter search spaces and prohibitively expensive training costs.

Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification

no code implementations22 Feb 2018 Max Simchowitz, Horia Mania, Stephen Tu, Michael. I. Jordan, Benjamin Recht

We prove that the ordinary least-squares (OLS) estimator attains nearly minimax optimal performance for the identification of linear dynamical systems from a single observed trajectory.

Time Series Time Series Analysis

Simple random search provides a competitive approach to reinforcement learning

25 code implementations19 Mar 2018 Horia Mania, Aurelia Guy, Benjamin Recht

A common belief in model-free reinforcement learning is that methods based on random search in the parameter space of policies exhibit significantly worse sample complexity than those that explore the space of actions.

Computational Efficiency Continuous Control +2

Finite-Data Performance Guarantees for the Output-Feedback Control of an Unknown System

1 code implementation25 Mar 2018 Ross Boczar, Nikolai Matni, Benjamin Recht

As the systems we control become more complex, first-principle modeling becomes either impossible or intractable, motivating the use of machine learning techniques for the control of systems with continuous action spaces.

Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law

no code implementations4 Apr 2018 Max Simchowitz, Ahmed El Alaoui, Benjamin Recht

We show that for every $\mathtt{gap} \in (0, 1/2]$, there exists a distribution over matrices $\mathbf{M}$ for which 1) $\mathrm{gap}_r(\mathbf{M}) = \Omega(\mathtt{gap})$ (where $\mathrm{gap}_r(\mathbf{M})$ is the normalized gap between the $r$ and $r+1$-st largest-magnitude eigenvector of $\mathbf{M}$), and 2) any algorithm $\mathsf{Alg}$ which takes fewer than $\mathrm{const} \times \frac{r \log d}{\sqrt{\mathtt{gap}}}$ queries fails (with overwhelming probability) to identity a matrix $\widehat{\mathsf{V}} \in \mathbb{R}^{d \times r}$ with orthonormal columns for which $\langle \widehat{\mathsf{V}}, \mathbf{M} \widehat{\mathsf{V}}\rangle \ge (1 - \mathrm{const} \times \mathtt{gap})\sum_{i=1}^r \lambda_i(\mathbf{M})$.

Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulator

no code implementations NeurIPS 2018 Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, Stephen Tu

We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs.

Do CIFAR-10 Classifiers Generalize to CIFAR-10?

3 code implementations1 Jun 2018 Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar

Although we ensure that the new test set is as close to the original data distribution as possible, we find a large drop in accuracy (4% to 10%) for a broad range of deep learning models.

A Tour of Reinforcement Learning: The View from Continuous Control

1 code implementation25 Jun 2018 Benjamin Recht

This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications.

Continuous Control Learning Theory +2

Safely Learning to Control the Constrained Linear Quadratic Regulator

2 code implementations26 Sep 2018 Sarah Dean, Stephen Tu, Nikolai Matni, Benjamin Recht

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques.

A Successive-Elimination Approach to Adaptive Robotic Sensing

no code implementations27 Sep 2018 Esther Rolf, David Fridovich-Keil, Max Simchowitz, Benjamin Recht, Claire Tomlin

We study an adaptive source seeking problem, in which a mobile robot must identify the strongest emitter(s) of a signal in an environment with background emissions.

Trajectory Planning

Minimax Lower Bounds for $\mathcal{H}_\infty$-Norm Estimation

no code implementations28 Sep 2018 Stephen Tu, Ross Boczar, Benjamin Recht

The problem of estimating the $\mathcal{H}_\infty$-norm of an LTI system from noisy input/output measurements has attracted recent attention as an alternative to parameter identification for bounding unmodeled dynamics in robust control.

Simple random search of static linear policies is competitive for reinforcement learning

1 code implementation NeurIPS 2018 Horia Mania, Aurelia Guy, Benjamin Recht

Common evaluation methodology shows that our method matches state-of-the-art sample efficiency on the benchmark MuJoCo locomotion tasks.

Continuous Control reinforcement-learning +1

Learning Linear Dynamical Systems with Semi-Parametric Least Squares

1 code implementation2 Feb 2019 Max Simchowitz, Ross Boczar, Benjamin Recht

We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning.

Active Learning

Certainty Equivalence is Efficient for Linear Quadratic Control

no code implementations NeurIPS 2019 Horia Mania, Stephen Tu, Benjamin Recht

We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system and the cost incurred by using the optimal LQ controller enjoys a fast statistical rate, scaling as the square of the parameter error.

Model Similarity Mitigates Test Set Overuse

no code implementations NeurIPS 2019 Horia Mania, John Miller, Ludwig Schmidt, Moritz Hardt, Benjamin Recht

Excessive reuse of test data has become commonplace in today's machine learning workflows.

Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulator

no code implementations NeurIPS 2019 Karl Krauth, Stephen Tu, Benjamin Recht

We study the sample complexity of approximate policy iteration (PI) for the Linear Quadratic Regulator (LQR), building on a recent line of work using LQR as a testbed to understand the limits of reinforcement learning (RL) algorithms on continuous control tasks.

Continuous Control Reinforcement Learning (RL)

Towards augmenting crisis counselor training by improving message retrieval

no code implementations WS 2019 Orianna Demasi, Marti A. Hearst, Benjamin Recht

A fundamental challenge when training counselors is presenting novices with the opportunity to practice counseling distressed individuals without exacerbating a situation.

Retrieval

Do Image Classifiers Generalize Across Time?

1 code implementation ICCV 2021 Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, Ludwig Schmidt

Additionally, we evaluate three detection models and show that natural perturbations induce both classification as well as localization errors, leading to a median drop in detection mAP of 14 points.

General Classification Video Object Detection

Robust Guarantees for Perception-Based Control

no code implementations L4DC 2020 Sarah Dean, Nikolai Matni, Benjamin Recht, Vickie Ye

Motivated by vision-based control of autonomous vehicles, we consider the problem of controlling a known linear dynamical system for which partial state information, such as vehicle position, is extracted from complex and nonlinear data, such as a camera image.

Autonomous Vehicles Position

A Meta-Analysis of Overfitting in Machine Learning

no code implementations NeurIPS 2019 Rebecca Roelofs, Vaishaal Shankar, Benjamin Recht, Sara Fridovich-Keil, Moritz Hardt, John Miller, Ludwig Schmidt

By systematically comparing the public ranking with the final ranking, we assess how much participants adapted to the holdout set over the course of a competition.

BIG-bench Machine Learning Holdout Set

Recommendations and User Agency: The Reachability of Collaboratively-Filtered Information

2 code implementations20 Dec 2019 Sarah Dean, Sarah Rich, Benjamin Recht

When the systems are deployed, these models determine the availability of content and information to different users.

Recommendation Systems

Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information

1 code implementation12 Mar 2020 Esther Rolf, Michael. I. Jordan, Benjamin Recht

Observational data are often accompanied by natural structural indices, such as time stamps or geographic locations, which are meaningful to prediction tasks but are often discarded.

BIG-bench Machine Learning

The Effect of Natural Distribution Shift on Question Answering Models

no code implementations ICML 2020 John Miller, Karl Krauth, Benjamin Recht, Ludwig Schmidt

We build four new test sets for the Stanford Question Answering Dataset (SQuAD) and evaluate the ability of question-answering systems to generalize to new data.

Question Answering

Active Learning for Nonlinear System Identification with Guarantees

no code implementations18 Jun 2020 Horia Mania, Michael. I. Jordan, Benjamin Recht

While the identification of nonlinear dynamical systems is a fundamental building block of model-based reinforcement learning and feedback control, its sample complexity is only understood for systems that either have discrete states and actions or for systems that can be identified from data generated by i. i. d.

Active Learning Model-based Reinforcement Learning +1

Certainty Equivalent Perception-Based Control

1 code implementation27 Aug 2020 Sarah Dean, Benjamin Recht

In order to certify performance and safety, feedback control requires precise characterization of sensor errors.

Autonomous Driving regression

A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagery

no code implementations16 Oct 2020 Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, Solomon Hsiang

Combining satellite imagery with machine learning (SIML) has the potential to address global challenges by remotely estimating socioeconomic and environmental conditions in data-poor regions, yet the resource requirements of SIML limit its accessibility and use.

BIG-bench Machine Learning regression +1

Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions

1 code implementation30 Oct 2020 Sarah Dean, Andrew J. Taylor, Ryan K. Cosner, Benjamin Recht, Aaron D. Ames

The guarantees ensured by these controllers often rely on accurate estimates of the system state for determining control actions.

Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertainty

no code implementations21 Nov 2020 Andrew J. Taylor, Victor D. Dorobantu, Sarah Dean, Benjamin Recht, Yisong Yue, Aaron D. Ames

Modern nonlinear control theory seeks to endow systems with properties such as stability and safety, and has been deployed successfully across various domains.

Interpolating Classifiers Make Few Mistakes

no code implementations28 Jan 2021 Tengyuan Liang, Benjamin Recht

Under the assumption that the data is independently and identically distributed, the mistake bound implies that MNIC generalizes at a rate proportional to the norm of the interpolating solution and inversely proportional to the number of data points.

Patterns, predictions, and actions: A story about machine learning

no code implementations10 Feb 2021 Moritz Hardt, Benjamin Recht

This graduate textbook on machine learning tells a story of how patterns in data support predictions and consequential actions.

BIG-bench Machine Learning Causal Inference +3

Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data

1 code implementation5 Mar 2021 Esther Rolf, Theodora Worledge, Benjamin Recht, Michael I. Jordan

Collecting more diverse and representative training data is often touted as a remedy for the disparate performance of machine learning predictors across subpopulations.

Quantifying Availability and Discovery in Recommender Systems via Stochastic Reachability

1 code implementation30 Jun 2021 Mihaela Curmei, Sarah Dean, Benjamin Recht

In this work, we consider how preference models in interactive recommendation systems determine the availability of content and users' opportunities for discovery.

Recommendation Systems

Towards Psychologically-Grounded Dynamic Preference Models

no code implementations1 Aug 2022 Mihaela Curmei, Andreas Haupt, Dylan Hadfield-Menell, Benjamin Recht

Second, we discuss implications of dynamic preference models for recommendation systems evaluation and design.

Recommendation Systems

Gradient Descent Provably Solves Nonlinear Tomographic Reconstruction

no code implementations6 Oct 2023 Sara Fridovich-Keil, Fabrizio Valdivia, Gordon Wetzstein, Benjamin Recht, Mahdi Soltanolkotabi

We show that this approach reduces metal artifacts compared to a commercial reconstruction of a human skull with metal dental crowns.

Computed Tomography (CT)

Cannot find the paper you are looking for? You can Submit a new open access paper.