Search Results for author:
Found 24 papers, 9 papers with code
Automatic Gradient Descent: Deep Learning without Hyperparameters
Automatic gradient descent trains both fully-connected and convolutional networks out-of-the-box and at ImageNet scale.
Adaptive-Control-Oriented Meta-Learning for Nonlinear Systems
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Control-oriented meta-learning
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks
In order to safely deploy Deep Neural Networks (DNNs) within the perception pipelines of real-time decision making systems, there is a need for safeguards that can detect out-of-training-distribution (OoD) inputs both efficiently and accurately.
Learning Control-Oriented Dynamical Structure from Data
Even for known nonlinear dynamical systems, feedback controller synthesis is a difficult problem that often requires leveraging the particular structure of the dynamics to induce a stable closed-loop system.
Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization
Most modern learning problems are highly overparameterized, meaning that there are many more parameters than the number of training data points, and as a result, the training loss may have infinitely many global minima (parameter vectors that perfectly interpolate the training data).
Online Learning for Traffic Routing under Unknown Preferences
at each period, we show that our approach obtains an expected regret and road capacity violation of $O(\sqrt{T})$, where $T$ is the number of periods over which tolls are updated.
Uncertainty-Aware Meta-Learning for Multimodal Task Distributions
In this work, we present UnLiMiTD (uncertainty-aware meta-learning for multimodal task distributions), a novel method for meta-learning that (1) makes probabilistic predictions on in-distribution tasks efficiently, (2) is capable of detecting OoD context data at test time, and (3) performs on heterogeneous, multimodal task distributions.
SketchOGD: Memory-Efficient Continual Learning
This paper proposes a memory-efficient solution to catastrophic forgetting, improving upon an established algorithm known as orthogonal gradient descent (OGD).
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization
In an attempt to shed some light on why this is the case, we revisit some minimax properties of stochastic gradient descent (SGD) for the square loss of linear models---originally developed in the 1990's---and extend them to general stochastic mirror descent (SMD) algorithms for general loss functions and nonlinear models.
A Stochastic Interpretation of Stochastic Mirror Descent: Risk-Sensitive Optimality
Stochastic mirror descent (SMD) is a fairly new family of algorithms that has recently found a wide range of applications in optimization, machine learning, and control.
Orthogonal Gradient Descent for Continual Learning
In this paper, we propose to address this issue from a parameter space perspective and study an approach to restrict the direction of the gradient updates to avoid forgetting previously-learned data.
Stochastic Mirror Descent on Overparameterized Nonlinear Models
On the theory side, we show that in the overparameterized nonlinear setting, if the initialization is close enough to the manifold of global optima, SMD with sufficiently small step size converges to a global minimum that is approximately the closest global minimum in Bregman divergence, thus attaining approximate implicit regularization.
Explicit Regularization via Regularizer Mirror Descent
RMD starts with a standard cost which is the sum of the training loss and a convex regularizer of the weights.
A Unified View of SDP-based Neural Network Verification through Completely Positive Programming
Verifying that input-output relationships of a neural network conform to prescribed operational specifications is a key enabler towards deploying these networks in safety-critical applications.
Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently
Driven by the empirical success and wide use of deep neural networks, understanding the generalization performance of overparameterized models has become an increasingly popular question.
Uncertainty in Contrastive Learning: On the Predictability of Downstream Performance
In this work, we study whether the uncertainty of such a representation can be quantified for a single datapoint in a meaningful way.
One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares
While deep neural networks are capable of achieving state-of-the-art performance in various domains, their training typically requires iterating for many passes over the dataset.
Data-Driven Control with Inherent Lyapunov Stability
Recent advances in learning-based control leverage deep function approximators, such as neural networks, to model the evolution of controlled dynamical systems over time.
Online Learning for Equilibrium Pricing in Markets under Incomplete Information
In this incomplete information setting, we consider the online learning problem of learning equilibrium prices over time while jointly optimizing three performance metrics -- unmet demand, cost regret, and payment regret -- pertinent in the context of equilibrium pricing over a horizon of $T$ periods.
On the Effects of Data Heterogeneity on the Convergence Rates of Distributed Linear System Solvers
Using this notion, we bound and compare the convergence rates of the studied algorithms and capture the effects of both cross-machine and local data heterogeneity on these quantities.
Representation Reliability and Its Impact on Downstream Tasks
Self-supervised pre-trained models extract general-purpose representations from data, and quantifying how reliable they are is crucial because many downstream models use these representations as input for their own tasks.
A Unified Approach to Controlling Implicit Regularization via Mirror Descent
However, the implicit regularization of different algorithms are confined to either a specific geometry or a particular class of learning problems, indicating a gap in a general approach for controlling the implicit regularization.
Private Synthetic Data Meets Ensemble Learning
When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and real data.
