Search Results for author:
Found 41 papers, 2 papers with code
Dynamic Regret Analysis of Safe Distributed Online Optimization for Convex and Non-convex Problems
We prove that for convex functions, D-Safe-OGD achieves a dynamic regret bound of $O(T^{2/3} \sqrt{\log T} + T^{1/3}C_T^*)$, where $C_T^*$ denotes the path-length of the best minimizer sequence.
TAP: The Attention Patch for Cross-Modal Knowledge Transfer from Unlabeled Data
This work investigates the intersection of cross modal learning and semi supervised learning, where we aim to improve the supervised learning performance of the primary modality by borrowing missing information from an unlabeled modality.
On the Stability Analysis of Open Federated Learning Systems
We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process.
Distributed Online System Identification for LTI Systems Using Reverse Experience Replay
Inspired by this work, we study distributed online system identification of LTI systems over a multi-agent network.
TAKDE: Temporal Adaptive Kernel Density Estimator for Real-Time Dynamic Density Estimation
Kernel density estimation is arguably one of the most commonly used density estimation techniques, and the use of "sliding window" mechanism adapts kernel density estimators to dynamic processes.
A Sparse Expansion For Deep Gaussian Processes
In this work, we use Deep Gaussian Processes (DGPs) as statistical surrogates for stochastic processes with complex distributions.
On Centralized and Distributed Mirror Descent: Convergence Analysis Using Quadratic Constraints
Our numerical experiments on strongly convex problems indicate that our framework certifies superior convergence rates compared to the existing rates for distributed GD.
Regret Analysis of Distributed Online LQR Control for Unknown LTI Systems
Consider a multi-agent network where each agent is modeled as a LTI system.
Decentralized Riemannian Gradient Descent on the Stiefel Manifold
The global function is represented as a finite sum of smooth local functions, where each local function is associated with one agent and agents communicate with each other over an undirected connected graph.
On the Local Linear Rate of Consensus on the Stiefel Manifold
We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold.
Linear Convergence of Distributed Mirror Descent with Integral Feedback for Strongly Convex Problems
Distributed optimization often requires finding the minimum of a global objective function written as a sum of local functions.
Distributed Online Linear Quadratic Control for Linear Time-invariant Systems
Recent advancement in online optimization and control has provided novel tools to study LQ problems that are robust to time-varying cost parameters.
Distributed Mirror Descent with Integral Feedback: Asymptotic Convergence Analysis of Continuous-time Dynamics
This work addresses distributed optimization, where a network of agents wants to minimize a global strongly convex objective function.
Unconstrained Online Optimization: Dynamic Regret Analysis of Strongly Convex and Smooth Problems
The regret bound of dynamic online learning algorithms is often expressed in terms of the variation in the function sequence ($V_T$) and/or the path-length of the minimizer sequence after $T$ rounds.
High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces
Density estimation plays a key role in many tasks in machine learning, statistical inference, and visualization.
Learning from Non-Random Data in Hilbert Spaces: An Optimal Recovery Perspective
Then, for any Hilbert space, we show that Optimal Recovery provides a formula which is user-friendly from an algorithmic point-of-view, as long as the hypothesis class is linear.
On Distributed Non-convex Optimization: Projected Subgradient Method For Weakly Convex Problems in Networks
In this paper, we propose a distributed implementation of the stochastic subgradient method with a theoretical guarantee.
Statistical and Topological Properties of Sliced Probability Divergences
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base divergence' between one-dimensional random projections of the two measures.
Generalized Sliced Distances for Probability Distributions
Probability metrics have become an indispensable part of modern statistics and machine learning, and they play a quintessential role in various applications, including statistical hypothesis testing and generative modeling.
RFN: A Random-Feature Based Newton Method for Empirical Risk Minimization in Reproducing Kernel Hilbert Spaces
Large-scale finite-sum problems can be solved using efficient variants of Newton method, where the Hessian is approximated via sub-samples of data.
Generalization Guarantees for Sparse Kernel Approximation with Entropic Optimal Features
Despite their success, kernel methods suffer from a massive computational cost in practice.
ORCCA: Optimal Randomized Canonical Correlation Analysis
We prove that this method, called ORCCA, can outperform (in expectation) the corresponding Kernel CCA with a default kernel.
A Mean-Field Theory for Kernel Alignment with Random Features in Generative and Discriminative Models
We subsequently leverage a particle stochastic gradient descent (SGD) method to solve the derived finite dimensional optimization problem.
A Mean-Field Theory for Kernel Alignment with Random Features in Generative Adverserial Networks
We subsequently leverage a particle stochastic gradient descent (SGD) method to solve finite dimensional optimization problems.
Distributed Parameter Estimation in Randomized One-hidden-layer Neural Networks
This paper addresses distributed parameter estimation in randomized one-hidden-layer neural networks.
On Sampling Random Features From Empirical Leverage Scores: Implementation and Theoretical Guarantees
Random features provide a practical framework for large-scale kernel approximation and supervised learning.
Learning Bounds for Greedy Approximation with Explicit Feature Maps from Multiple Kernels
We establish an out-of-sample error bound capturing the trade-off between the error in terms of explicit features (approximation error) and the error due to spectral properties of the best model in the Hilbert space associated to the combined kernel (spectral error).
On Data-Dependent Random Features for Improved Generalization in Supervised Learning
The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning.
On Optimal Generalizability in Parametric Learning
Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance.
Nonlinear Sequential Accepts and Rejects for Identification of Top Arms in Stochastic Bandits
At each round, the budget is divided by a nonlinear function of remaining arms, and the arms are pulled correspondingly.
An Online Optimization Approach for Multi-Agent Tracking of Dynamic Parameters in the Presence of Adversarial Noise
We formulate this problem as a distributed online optimization where agents communicate with each other to track the minimizer of the global loss.
Distributed Online Optimization in Dynamic Environments Using Mirror Descent
A network of agents aim to track the minimizer of a global time-varying convex function.
On Sequential Elimination Algorithms for Best-Arm Identification in Multi-Armed Bandits
Based on this result, we develop an algorithm that divides the budget according to a nonlinear function of remaining arms at each round.
Online Optimization in Dynamic Environments: Improved Regret Rates for Strongly Convex Problems
In this paper, we address tracking of a time-varying parameter with unknown dynamics.
Distributed Estimation of Dynamic Parameters : Regret Analysis
To this end, we use a notion of dynamic regret which suits the online, non-stationary nature of the problem.
Learning without Recall by Random Walks on Directed Graphs
Each agent might not be able to distinguish the true state based only on her private observations.
Switching to Learn
A network of agents attempt to learn some unknown state of the world drawn by nature from a finite set.
Online Optimization : Competing with Dynamic Comparators
Recent literature on online learning has focused on developing adaptive algorithms that take advantage of a regularity of the sequence of observations, yet retain worst-case performance guarantees.
Distributed Detection : Finite-time Analysis and Impact of Network Topology
In contrast to the existing literature which focuses on asymptotic learning, we provide a finite-time analysis.
Online Learning of Dynamic Parameters in Social Networks
Based on the decomposition of the global loss function, we introduce two update mechanisms, each of which generates an estimate of the true state.
Exponentially Fast Parameter Estimation in Networks Using Distributed Dual Averaging
When the true state is globally identifiable, and the network is connected, we prove that agents eventually learn the true parameter using a randomized gossip scheme.
