Search Results for author:
Found 27 papers, 10 papers with code
Learning Chaotic Dynamics with Embedded Dissipativity
More specifically, by leveraging control-theoretic ideas, we derive algebraic conditions based on the learned energy-like function that ensure asymptotic convergence to an invariant level set.
Meta-Learning for Adaptive Control with Automated Mirror Descent
In this paper, we propose a novel method that combines meta-learning and adaptation laws based on mirror descent, a popular generalization of gradient descent, which takes advantage of the potentially non-Euclidean geometry of the parameter space.
Adapting Differentially Private Synthetic Data to Relational Databases
Existing differentially private (DP) synthetic data generation mechanisms typically assume a single-source table.
Private Synthetic Data Meets Ensemble Learning
When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and real data.
A Unified Approach to Controlling Implicit Regularization via Mirror Descent
However, the implicit regularization of different algorithms are confined to either a specific geometry or a particular class of learning problems, indicating a gap in a general approach for controlling the implicit regularization.
Quantifying Representation Reliability in Self-Supervised Learning Models
Quantifying the reliability of these representations is crucial, as many downstream models rely on them as input for their own tasks.
SketchOGD: Memory-Efficient Continual Learning
This paper proposes a memory-efficient solution to catastrophic forgetting, improving upon an established algorithm known as orthogonal gradient descent (OGD).
On the Effects of Data Heterogeneity on the Convergence Rates of Distributed Linear System Solvers
We consider the problem of solving a large-scale system of linear equations in a distributed or federated manner by a taskmaster and a set of machines, each possessing a subset of the equations.
Automatic Gradient Descent: Deep Learning without Hyperparameters
Automatic gradient descent trains both fully-connected and convolutional networks out-of-the-box and at ImageNet scale.
Online Learning for Equilibrium Pricing in Markets under Incomplete Information
In this incomplete information setting, we consider the online learning problem of learning equilibrium prices over time while jointly optimizing three performance metrics -- unmet demand, cost regret, and payment regret -- pertinent in the context of equilibrium pricing over a horizon of $T$ periods.
Data-Driven Control with Inherent Lyapunov Stability
Recent advances in learning-based control leverage deep function approximators, such as neural networks, to model the evolution of controlled dynamical systems over time.
Learning Control-Oriented Dynamical Structure from Data
Even for known nonlinear dynamical systems, feedback controller synthesis is a difficult problem that often requires leveraging the particular structure of the dynamics to induce a stable closed-loop system.
Uncertainty-Aware Meta-Learning for Multimodal Task Distributions
In this work, we present UnLiMiTD (uncertainty-aware meta-learning for multimodal task distributions), a novel method for meta-learning that (1) makes probabilistic predictions on in-distribution tasks efficiently, (2) is capable of detecting OoD context data at test time, and (3) performs on heterogeneous, multimodal task distributions.
One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares
While deep neural networks are capable of achieving state-of-the-art performance in various domains, their training typically requires iterating for many passes over the dataset.
Uncertainty in Contrastive Learning: On the Predictability of Downstream Performance
In this work, we study whether the uncertainty of such a representation can be quantified for a single datapoint in a meaningful way.
Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently
Driven by the empirical success and wide use of deep neural networks, understanding the generalization performance of overparameterized models has become an increasingly popular question.
Control-oriented meta-learning
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Online Learning for Traffic Routing under Unknown Preferences
at each period, we show that our approach obtains an expected regret and road capacity violation of $O(\sqrt{T})$, where $T$ is the number of periods over which tolls are updated.
A Unified View of SDP-based Neural Network Verification through Completely Positive Programming
Verifying that input-output relationships of a neural network conform to prescribed operational specifications is a key enabler towards deploying these networks in safety-critical applications.
Explicit Regularization via Regularizer Mirror Descent
RMD starts with a standard cost which is the sum of the training loss and a convex regularizer of the weights.
Adaptive-Control-Oriented Meta-Learning for Nonlinear Systems
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks
In order to safely deploy Deep Neural Networks (DNNs) within the perception pipelines of real-time decision making systems, there is a need for safeguards that can detect out-of-training-distribution (OoD) inputs both efficiently and accurately.
Orthogonal Gradient Descent for Continual Learning
In this paper, we propose to address this issue from a parameter space perspective and study an approach to restrict the direction of the gradient updates to avoid forgetting previously-learned data.
Stochastic Mirror Descent on Overparameterized Nonlinear Models
On the theory side, we show that in the overparameterized nonlinear setting, if the initialization is close enough to the manifold of global optima, SMD with sufficiently small step size converges to a global minimum that is approximately the closest global minimum in Bregman divergence, thus attaining approximate implicit regularization.
Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization
Most modern learning problems are highly overparameterized, meaning that there are many more parameters than the number of training data points, and as a result, the training loss may have infinitely many global minima (parameter vectors that perfectly interpolate the training data).
A Stochastic Interpretation of Stochastic Mirror Descent: Risk-Sensitive Optimality
Stochastic mirror descent (SMD) is a fairly new family of algorithms that has recently found a wide range of applications in optimization, machine learning, and control.
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization
In an attempt to shed some light on why this is the case, we revisit some minimax properties of stochastic gradient descent (SGD) for the square loss of linear models---originally developed in the 1990's---and extend them to general stochastic mirror descent (SMD) algorithms for general loss functions and nonlinear models.
