Search Results for author:
Found 54 papers, 9 papers with code
Kernel Conditional Moment Constraints for Confounding Robust Inference
It can be shown that our estimator contains the recently proposed sharp estimator by Dorn and Guo (2022) as a special case, and our method enables a novel extension of the classical marginal sensitivity model using f-divergence.
Robust Knowledge Transfer in Tiered Reinforcement Learning
In this paper, we study the Tiered Reinforcement Learning setting, a parallel transfer learning framework, where the goal is to transfer knowledge from the low-tier (source) task to the high-tier (target) task to reduce the exploration risk of the latter while solving the two tasks in parallel.
Stochastic Policy Gradient Methods: Improved Sample Complexity for Fisher-non-degenerate Policies
Recently, the impressive empirical success of policy gradient (PG) methods has catalyzed the development of their theoretical foundations.
Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games
Instead, we show that $N$ agents running policy mirror ascent converge to the Nash equilibrium of the regularized game within $\tilde{\mathcal{O}}(\varepsilon^{-2})$ samples from a single sample trajectory without a population generative model, up to a standard $\mathcal{O}(\frac{1}{\sqrt{N}})$ error due to the mean field.
Learning to Optimize with Stochastic Dominance Constraints
In real-world decision-making, uncertainty is important yet difficult to handle.
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
Adaptive gradient methods have shown their ability to adjust the stepsizes on the fly in a parameter-agnostic manner, and empirically achieve faster convergence for solving minimization problems.
Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm
Natural actor-critic (NAC) and its variants, equipped with the representation power of neural networks, have demonstrated impressive empirical success in solving Markov decision problems with large state spaces.
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
Adaptive algorithms like AdaGrad and AMSGrad are successful in nonconvex optimization owing to their parameter-agnostic ability -- requiring no a priori knowledge about problem-specific parameters nor tuning of learning rates.
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization
We provide a general framework for solving differentially private stochastic minimax optimization (DP-SMO) problems, which enables the practitioners to bring their own base optimization algorithm and use it as a black-box to obtain the near-optimal privacy-loss trade-off.
Generalization Bounds of Nonconvex-(Strongly)-Concave Stochastic Minimax Optimization
We further study the algorithm-dependent generalization bounds via stability arguments of algorithms.
Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function
We prove that the total sample complexity of SCRN in achieving $\epsilon$-global optimum is $\mathcal{O}(\epsilon^{-7/(2\alpha)+1})$ for $1\le\alpha< 3/2$ and $\mathcal{\tilde{O}}(\epsilon^{-2/(\alpha)})$ for $3/2\le\alpha\le 2$.
Momentum-Based Policy Gradient with Second-Order Information
SHARP algorithm is parameter-free, achieving $\epsilon$-approximate first-order stationary point with $O(\epsilon^{-3})$ number of trajectories, while using a batch size of $O(1)$ at each iteration.
Learning to Control Partially Observed Systems with Finite Memory
We consider the reinforcement learning problem for partially observed Markov decision processes (POMDPs) with large or even countably infinite state spaces, where the controller has access to only noisy observations of the underlying controlled Markov chain.
Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimization
We also provide a direct single-loop algorithm, using the LPD method, that achieves the iteration complexity of $O(\sqrt{\frac{L_x}{\varepsilon}} + \frac{\|A\|}{\sqrt{\mu_y \varepsilon}} + \sqrt{\frac{L_y}{\varepsilon}})$.
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity
Gradient descent ascent (GDA), the simplest single-loop algorithm for nonconvex minimax optimization, is widely used in practical applications such as generative adversarial networks (GANs) and adversarial training.
On the Bias-Variance-Cost Tradeoff of Stochastic Optimization
We consider stochastic optimization when one only has access to biased stochastic oracles of the objective, and obtaining stochastic gradients with low biases comes at high costs.
Sample-efficient actor-critic algorithms with an etiquette for zero-sum Markov games
Our sample complexities also match the best-known results for global convergence of policy gradient and two time-scale actor-critic algorithms in the single agent setting.
Efficient Wasserstein and Sinkhorn Policy Optimization
Trust-region methods based on Kullback-Leibler divergence are pervasively used to stabilize policy optimization in reinforcement learning.
Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
Furthermore, under mild regularity conditions on the concentrability coefficient and basis vectors, we prove that entropy-regularized NPG exhibits \emph{linear convergence} up to a function approximation error.
The Complexity of Nonconvex-Strongly-Concave Minimax Optimization
In the averaged smooth finite-sum setting, our proposed algorithm improves over previous algorithms by providing a nearly-tight dependence on the condition number.
Simulation Studies on Deep Reinforcement Learning for Building Control with Human Interaction
The building sector consumes the largest energy in the world, and there have been considerable research interests in energy consumption and comfort management of buildings.
Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation
In this paper, we study the dynamics of temporal difference learning with neural network-based value function approximation over a general state space, namely, \emph{Neural TD learning}.
A Discrete-Time Switching System Analysis of Q-learning
Based on these two systems, we derive a new finite-time error bound of asynchronous Q-learning when a constant stepsize is used.
Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax Problems
Nonconvex minimax problems appear frequently in emerging machine learning applications, such as generative adversarial networks and adversarial learning.
A Unified Switching System Perspective and Convergence Analysis of Q-Learning Algorithms
This paper develops a novel and unified framework to analyze the convergence of a large family of Q-learning algorithms from the switching system perspective.
The Devil is in the Detail: A Framework for Macroscopic Prediction via Microscopic Models
Macroscopic data aggregated from microscopic events are pervasive in machine learning, such as country-level COVID-19 infection statistics based on city-level data.
A Catalyst Framework for Minimax Optimization
We introduce a generic \emph{two-loop} scheme for smooth minimax optimization with strongly-convex-concave objectives.
Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learning
Conditional stochastic optimization covers a variety of applications ranging from invariant learning and causal inference to meta-learning.
The Mean-Squared Error of Double Q-Learning
In this paper, we establish a theoretical comparison between the asymptotic mean-squared error of Double Q-learning and Q-learning.
Biased Stochastic Gradient Descent for Conditional Stochastic Optimization
Conditional Stochastic Optimization (CSO) covers a variety of applications ranging from meta-learning and causal inference to invariant learning.
Periodic Q-Learning
The use of target networks is a common practice in deep reinforcement learning for stabilizing the training; however, theoretical understanding of this technique is still limited.
Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems
Nonconvex minimax problems appear frequently in emerging machine learning applications, such as generative adversarial networks and adversarial learning.
A Unified Switching System Perspective and O.D.E. Analysis of Q-Learning Algorithms
In this paper, we introduce a unified framework for analyzing a large family of Q-learning algorithms, based on switching system perspectives and ODE-based stochastic approximation.
Learning Positive Functions with Pseudo Mirror Descent
The nonparametric learning of positive-valued functions appears widely in machine learning, especially in the context of estimating intensity functions of point processes.
Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents
This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate.
Distributed Optimization
Multi-agent Reinforcement Learning
+2
Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization
In this paper, we study a class of stochastic optimization problems, referred to as the \emph{Conditional Stochastic Optimization} (CSO), in the form of $\min_{x \in \mathcal{X}} \EE_{\xi}f_\xi\Big({\EE_{\eta|\xi}[g_\eta(x,\xi)]}\Big)$, which finds a wide spectrum of applications including portfolio selection, reinforcement learning, robust learning, causal inference and so on.
Exponential Family Estimation via Adversarial Dynamics Embedding
We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks.
Target-Based Temporal Difference Learning
The use of target networks has been a popular and key component of recent deep Q-learning algorithms for reinforcement learning, yet little is known from the theory side.
Quadratic Decomposable Submodular Function Minimization: Theory and Practice (Computation and Analysis of PageRank over Hypergraphs)
We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization (QDSFM), which allows to model a number of learning tasks on graphs and hypergraphs.
Predictive Approximate Bayesian Computation via Saddle Points
Approximate Bayesian computation (ABC) is an important methodology for Bayesian inference when the likelihood function is intractable.
Coupled Variational Bayes via Optimization Embedding
This flexible function class couples the variational distribution with the original parameters in the graphical models, allowing end-to-end learning of the graphical models by back-propagation through the variational distribution.
Kernel Exponential Family Estimation via Doubly Dual Embedding
We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space.
Quadratic Decomposable Submodular Function Minimization
The problem is closely related to decomposable submodular function minimization and arises in many learning on graphs and hypergraphs settings, such as graph-based semi-supervised learning and PageRank.
Nonparametric Hawkes Processes: Online Estimation and Generalization Bounds
In this paper, we design a nonparametric online algorithm for estimating the triggering functions of multivariate Hawkes processes.
Boosting the Actor with Dual Critic
This paper proposes a new actor-critic-style algorithm called Dual Actor-Critic or Dual-AC.
SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
When function approximation is used, solving the Bellman optimality equation with stability guarantees has remained a major open problem in reinforcement learning for decades.
Online Learning for Multivariate Hawkes Processes
We develop a nonparametric and online learning algorithm that estimates the triggering functions of a multivariate Hawkes process (MHP).
Stochastic Generative Hashing
Learning-based binary hashing has become a powerful paradigm for fast search and retrieval in massive databases.
Fast and Simple Optimization for Poisson Likelihood Models
Since almost all gradient-based optimization algorithms rely on Lipschitz-continuity, optimizing Poisson likelihood models with a guarantee of convergence can be challenging, especially for large-scale problems.
Learning from Conditional Distributions via Dual Embeddings
In such problems, each sample $x$ itself is associated with a conditional distribution $p(z|x)$ represented by samples $\{z_i\}_{i=1}^M$, and the goal is to learn a function $f$ that links these conditional distributions to target values $y$.
Time-Sensitive Recommendation From Recurrent User Activities
By making personalized suggestions, a recommender system is playing a crucial role in improving the engagement of users in modern web-services.
Semi-proximal Mirror-Prox for Nonsmooth Composite Minimization
We propose a new first-order optimisation algorithm to solve high-dimensional non-smooth composite minimisation problems.
Provable Bayesian Inference via Particle Mirror Descent
Bayesian methods are appealing in their flexibility in modeling complex data and ability in capturing uncertainty in parameters.
Scalable Kernel Methods via Doubly Stochastic Gradients
The general perception is that kernel methods are not scalable, and neural nets are the methods of choice for nonlinear learning problems.
