Search Results for author:
Found 44 papers, 2 papers with code
Decentralized Riemannian Gradient Descent on the Stiefel Manifold
The global function is represented as a finite sum of smooth local functions, where each local function is associated with one agent and agents communicate with each other over an undirected connected graph.
Statistical and Topological Properties of Sliced Probability Divergences
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base divergence' between one-dimensional random projections of the two measures.
On Data-Dependent Random Features for Improved Generalization in Supervised Learning
The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning.
On Optimal Generalizability in Parametric Learning
Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance.
Nonlinear Sequential Accepts and Rejects for Identification of Top Arms in Stochastic Bandits
At each round, the budget is divided by a nonlinear function of remaining arms, and the arms are pulled correspondingly.
On Sequential Elimination Algorithms for Best-Arm Identification in Multi-Armed Bandits
Based on this result, we develop an algorithm that divides the budget according to a nonlinear function of remaining arms at each round.
An Online Optimization Approach for Multi-Agent Tracking of Dynamic Parameters in the Presence of Adversarial Noise
We formulate this problem as a distributed online optimization where agents communicate with each other to track the minimizer of the global loss.
Distributed Online Optimization in Dynamic Environments Using Mirror Descent
A network of agents aim to track the minimizer of a global time-varying convex function.
Online Optimization in Dynamic Environments: Improved Regret Rates for Strongly Convex Problems
In this paper, we address tracking of a time-varying parameter with unknown dynamics.
Distributed Estimation of Dynamic Parameters : Regret Analysis
To this end, we use a notion of dynamic regret which suits the online, non-stationary nature of the problem.
Learning without Recall by Random Walks on Directed Graphs
Each agent might not be able to distinguish the true state based only on her private observations.
Switching to Learn
A network of agents attempt to learn some unknown state of the world drawn by nature from a finite set.
Online Optimization : Competing with Dynamic Comparators
Recent literature on online learning has focused on developing adaptive algorithms that take advantage of a regularity of the sequence of observations, yet retain worst-case performance guarantees.
Distributed Detection : Finite-time Analysis and Impact of Network Topology
In contrast to the existing literature which focuses on asymptotic learning, we provide a finite-time analysis.
Online Learning of Dynamic Parameters in Social Networks
Based on the decomposition of the global loss function, we introduce two update mechanisms, each of which generates an estimate of the true state.
Exponentially Fast Parameter Estimation in Networks Using Distributed Dual Averaging
When the true state is globally identifiable, and the network is connected, we prove that agents eventually learn the true parameter using a randomized gossip scheme.
Learning Bounds for Greedy Approximation with Explicit Feature Maps from Multiple Kernels
We establish an out-of-sample error bound capturing the trade-off between the error in terms of explicit features (approximation error) and the error due to spectral properties of the best model in the Hilbert space associated to the combined kernel (spectral error).
On Sampling Random Features From Empirical Leverage Scores: Implementation and Theoretical Guarantees
Random features provide a practical framework for large-scale kernel approximation and supervised learning.
Distributed Parameter Estimation in Randomized One-hidden-layer Neural Networks
This paper addresses distributed parameter estimation in randomized one-hidden-layer neural networks.
A Mean-Field Theory for Kernel Alignment with Random Features in Generative and Discriminative Models
We subsequently leverage a particle stochastic gradient descent (SGD) method to solve the derived finite dimensional optimization problem.
ORCCA: Optimal Randomized Canonical Correlation Analysis
We prove that this method, called ORCCA, can outperform (in expectation) the corresponding Kernel CCA with a default kernel.
Generalization Guarantees for Sparse Kernel Approximation with Entropic Optimal Features
Despite their success, kernel methods suffer from a massive computational cost in practice.
RFN: A Random-Feature Based Newton Method for Empirical Risk Minimization in Reproducing Kernel Hilbert Spaces
Large-scale finite-sum problems can be solved using efficient variants of Newton method, where the Hessian is approximated via sub-samples of data.
Generalized Sliced Distances for Probability Distributions
Probability metrics have become an indispensable part of modern statistics and machine learning, and they play a quintessential role in various applications, including statistical hypothesis testing and generative modeling.
On Distributed Non-convex Optimization: Projected Subgradient Method For Weakly Convex Problems in Networks
In this paper, we propose a distributed implementation of the stochastic subgradient method with a theoretical guarantee.
Unconstrained Online Optimization: Dynamic Regret Analysis of Strongly Convex and Smooth Problems
The regret bound of dynamic online learning algorithms is often expressed in terms of the variation in the function sequence ($V_T$) and/or the path-length of the minimizer sequence after $T$ rounds.
Learning from Non-Random Data in Hilbert Spaces: An Optimal Recovery Perspective
Then, for any Hilbert space, we show that Optimal Recovery provides a formula which is user-friendly from an algorithmic point-of-view, as long as the hypothesis class is linear.
High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces
Density estimation plays a key role in many tasks in machine learning, statistical inference, and visualization.
Distributed Mirror Descent with Integral Feedback: Asymptotic Convergence Analysis of Continuous-time Dynamics
This work addresses distributed optimization, where a network of agents wants to minimize a global strongly convex objective function.
Distributed Online Linear Quadratic Control for Linear Time-invariant Systems
Recent advancement in online optimization and control has provided novel tools to study LQ problems that are robust to time-varying cost parameters.
Linear Convergence of Distributed Mirror Descent with Integral Feedback for Strongly Convex Problems
Distributed optimization often requires finding the minimum of a global objective function written as a sum of local functions.
On the Local Linear Rate of Consensus on the Stiefel Manifold
We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold.
Regret Analysis of Distributed Online LQR Control for Unknown LTI Systems
Consider a multi-agent network where each agent is modeled as a LTI system.
On Centralized and Distributed Mirror Descent: Convergence Analysis Using Quadratic Constraints
Our numerical experiments on strongly convex problems indicate that our framework certifies superior convergence rates compared to the existing rates for distributed GD.
A Mean-Field Theory for Kernel Alignment with Random Features in Generative Adverserial Networks
We subsequently leverage a particle stochastic gradient descent (SGD) method to solve finite dimensional optimization problems.
A Sparse Expansion For Deep Gaussian Processes
In this work, we use Deep Gaussian Processes (DGPs) as statistical surrogates for stochastic processes with complex distributions.
TAKDE: Temporal Adaptive Kernel Density Estimator for Real-Time Dynamic Density Estimation
Kernel density estimation is arguably one of the most commonly used density estimation techniques, and the use of "sliding window" mechanism adapts kernel density estimators to dynamic processes.
Distributed Online System Identification for LTI Systems Using Reverse Experience Replay
Inspired by this work, we study distributed online system identification of LTI systems over a multi-agent network.
On the Stability Analysis of Open Federated Learning Systems
We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process.
TAP: The Attention Patch for Cross-Modal Knowledge Transfer from Unlabeled Modality
This paper addresses a cross-modal learning framework, where the objective is to enhance the performance of supervised learning in the primary modality using an unlabeled, unpaired secondary modality.
Dynamic Regret Analysis of Safe Distributed Online Optimization for Convex and Non-convex Problems
We prove that for convex functions, D-Safe-OGD achieves a dynamic regret bound of $O(T^{2/3} \sqrt{\log T} + T^{1/3}C_T^*)$, where $C_T^*$ denotes the path-length of the best minimizer sequence.
Regret Analysis of Distributed Online Control for LTI Systems with Adversarial Disturbances
For the unknown dynamics case, we design a distributed explore-then-commit approach, where in the exploration phase all agents jointly learn the system dynamics, and in the learning phase our proposed control algorithm is applied using each agent system estimate.
Tracking Dynamic Gaussian Density with a Theoretically Optimal Sliding Window Approach
Dynamic density estimation is ubiquitous in many applications, including computer vision and signal processing.
Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG
In this work, we study online linear quadratic Gaussian problems with a given linear constraint imposed on the controller.
