Search Results for author:
Found 43 papers, 3 papers with code
The Performance Analysis of Generalized Margin Maximizers on Separable Data
The performance of hard-margin SVM has been recently analyzed in~\cite{montanari2019generalization, deng2019model}.
The Generalization Error of Stochastic Mirror Descent on Over-Parametrized Linear Models
This is best understood in linear over-parametrized models where it has been shown that the celebrated stochastic gradient descent (SGD) algorithm finds an interpolating solution that is closest in Euclidean distance to the initial weight vector.
Precise Asymptotic Analysis of Deep Random Feature Models
We provide exact asymptotic expressions for the performance of regression by an $L-$layer deep random feature (RF) model, where the input is mapped through multiple random embedding and non-linear activation functions.
Stochastic Mirror Descent in Average Ensemble Models
The stochastic mirror descent (SMD) algorithm is a general class of training algorithms, which includes the celebrated stochastic gradient descent (SGD), as a special case.
Thompson Sampling Achieves $\tilde O(\sqrt{T})$ Regret in Linear Quadratic Control
By carefully prescribing an early exploration strategy and a policy update rule, we show that TS achieves order-optimal regret in adaptive control of multidimensional stabilizable LQRs.
Optimal Competitive-Ratio Control
The key techniques that underpin our explicit solution is a reduction of the control problem to a Nehari problem, along with a novel factorization of the clairvoyant controller's cost.
Explicit Regularization via Regularizer Mirror Descent
RMD starts with a standard cost which is the sum of the training loss and a convex regularizer of the weights.
Online estimation and control with optimal pathlength regret
A natural goal when designing online learning algorithms for non-stationary environments is to bound the regret of the algorithm in terms of the temporal variation of the input sequence.
How to Query An Oracle? Efficient Strategies to Label Data
Empirical studies suggest that each triplet query takes an expert at most 50\% more time compared with a pairwise query, indicating the effectiveness of the proposed $k$-ary query schemes.
Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems
Using these guarantees, we design adaptive control algorithms for unknown ARX systems with arbitrary strongly convex or convex quadratic regulating costs.
Regret-optimal Estimation and Control
We consider estimation and control in linear time-varying dynamical systems from the perspective of regret minimization.
Regret-Optimal Full-Information Control
Motivated by learning theory, as a criterion for controller design we focus on regret, defined as the difference between the LQR cost of a causal controller (that has only access to past and current disturbances) and the LQR cost of a clairvoyant one (that has also access to future disturbances).
Manifold Optimization for High Accuracy Spatial Location Estimation Using Ultrasound Waves
This paper reports the design of a high-accuracy spatial location estimation method using ultrasound waves by exploiting the fixed geometry of the transmitters.
Regret-Optimal Filtering for Prediction and Estimation
For the important case of signals that can be described with a time-invariant state-space, we provide an explicit construction for the regret optimal filter in the estimation (causal) and the prediction (strictly-causal) regimes.
Stability and Identification of Random Asynchronous Linear Time-Invariant Systems
In this model, each state variable is updated randomly and asynchronously with some probability according to the underlying system dynamics.
Regret-optimal measurement-feedback control
We consider measurement-feedback control in linear dynamical systems from the perspective of regret minimization.
The Performance Analysis of Generalized Margin Maximizer (GMM) on Separable Data
We also provide a detailed study for three special cases: ($1$) $\ell_2$-GMM that is the max-margin classifier, ($2$) $\ell_1$-GMM which encourages sparsity, and ($3$) $\ell_{\infty}$-GMM which is often used when the parameter vector has binary entries.
Robustifying Binary Classification to Adversarial Perturbation
To this end, in this paper we consider the problem of binary classification with adversarial perturbations.
Regret-optimal control in dynamic environments
We consider control in linear time-varying dynamical systems from the perspective of regret minimization.
Reinforcement Learning with Fast Stabilization in Linear Dynamical Systems
In this work, we study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems.
Model-based Reinforcement Learning
reinforcement-learning
+1
Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems
We study the problem of system identification and adaptive control in partially observable linear dynamical systems.
Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting
We study the problem of adaptive control in partially observable linear quadratic Gaussian control systems, where the model dynamics are unknown a priori.
The Power of Linear Controllers in LQR Control
We also show that cost of the optimal offline linear policy converges to the cost of the optimal online policy as the time horizon grows large, and consequently the optimal offline linear policy incurs linear regret relative to the optimal offline policy, even in the optimistic setting where the noise is drawn i. i. d from a known distribution.
Differentially Quantized Gradient Methods
We introduce the principle we call Differential Quantization (DQ) that prescribes compensating the past quantization errors to direct the descent trajectory of a quantized algorithm towards that of its unquantized counterpart.
Regret Minimization in Partially Observable Linear Quadratic Control
We propose a novel way to decompose the regret and provide an end-to-end sublinear regret upper bound for partially observable linear quadratic control.
Stochastic Mirror Descent on Overparameterized Nonlinear Models
On the theory side, we show that in the overparameterized nonlinear setting, if the initialization is close enough to the manifold of global optima, SMD with sufficiently small step size converges to a global minimum that is approximately the closest global minimum in Bregman divergence, thus attaining approximate implicit regularization.
The Impact of Regularization on High-dimensional Logistic Regression
In both cases, we obtain explicit expressions for various performance metrics and can find the values of the regularizer parameter that optimizes the desired performance.
Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization
Most modern learning problems are highly overparameterized, meaning that there are many more parameters than the number of training data points, and as a result, the training loss may have infinitely many global minima (parameter vectors that perfectly interpolate the training data).
A Stochastic Interpretation of Stochastic Mirror Descent: Risk-Sensitive Optimality
Stochastic mirror descent (SMD) is a fairly new family of algorithms that has recently found a wide range of applications in optimization, machine learning, and control.
Stochastic Linear Bandits with Hidden Low Rank Structure
We modify the image classification task into the SLB setting and empirically show that, when a pre-trained DNN provides the high dimensional feature representations, deploying PSLB results in significant reduction of regret and faster convergence to an accurate model compared to state-of-art algorithm.
Learning without the Phase: Regularized PhaseMax Achieves Optimal Sample Complexity
The problem of estimating an unknown signal, $\mathbf x_0\in \mathbb R^n$, from a vector $\mathbf y\in \mathbb R^m$ consisting of $m$ magnitude-only measurements of the form $y_i=|\mathbf a_i\mathbf x_0|$, where $\mathbf a_i$'s are the rows of a known measurement matrix $\mathbf A$ is a classical problem known as phase retrieval.
Low-Rank Riemannian Optimization on Positive Semidefinite Stochastic Matrices with Applications to Graph Clustering
This paper develops a Riemannian optimization framework for solving optimization problems on the set of symmetric positive semidefinite stochastic matrices.
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization
In an attempt to shed some light on why this is the case, we revisit some minimax properties of stochastic gradient descent (SGD) for the square loss of linear models---originally developed in the 1990's---and extend them to general stochastic mirror descent (SMD) algorithms for general loss functions and nonlinear models.
A Universal Analysis of Large-Scale Regularized Least Squares Solutions
Precise expressions for the asymptotic performance of LASSO have been obtained for a number of different cases, in particular when the elements of the dictionary matrix are sampled independently from a Gaussian distribution.
Distributed Solution of Large-Scale Linear Systems via Accelerated Projection-Based Consensus
In this paper, we consider a common scenario in which a taskmaster intends to solve a large-scale system of linear equations by distributing subsets of the equations among a number of computing machines/cores.
Entropic Causality and Greedy Minimum Entropy Coupling
This framework requires the solution of a minimum entropy coupling problem: Given marginal distributions of m discrete random variables, each on n states, find the joint distribution with minimum entropy, that respects the given marginals.
Crowdsourced Clustering: Querying Edges vs Triangles
When a generative model for the data is available (and we consider a few of these) we determine the cost of a query by its entropy; when such models do not exist we use the average response time per query of the workers as a surrogate for the cost.
Entropic Causal Inference
We show that the problem of finding the exogenous variable with minimum entropy is equivalent to the problem of finding minimum joint entropy given $n$ marginal distributions, also known as minimum entropy coupling problem.
Fundamental Limits of Budget-Fidelity Trade-off in Label Crowdsourcing
The results are established by a joint source channel (de)coding scheme, which represent the query scheme and inference, over parallel noisy channels, which model workers with imperfect skill levels.
LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements
In this work, we considerably strengthen these results by obtaining explicit expressions for $\|\hat x-\mu x_0\|_2$, for the regularized Generalized-LASSO, that are asymptotically precise when $m$ and $n$ grow large.
Graph Clustering With Missing Data: Convex Algorithms and Analysis
We consider the problem of finding clusters in an unweighted graph, when the graph is partially observed.
The Squared-Error of Generalized LASSO: A Precise Analysis
The first LASSO estimator assumes a-priori knowledge of $f(x_0)$ and is given by $\arg\min_{x}\{{\|y-Ax\|_2}~\text{subject to}~f(x)\leq f(x_0)\}$.
