Search Results for author:
Found 30 papers, 2 papers with code
Aligning Agent Policy with Externalities: Reward Design via Bilevel RL
To mathematically encapsulate the problem of aligning RL policy optimization with such externalities, we consider a bilevel optimization problem and connect it to a principal-agent framework, where the principal specifies the broader goals and constraints of the system at the upper level and the agent solves a Markov Decision Process (MDP) at the lower level.
On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization
To achieve that, we propose a Natural Actor-Critic algorithm with 2-Layer critic parametrization (NAC2L).
iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning
Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations.
Ada-NAV: Adaptive Trajectory-Based Sample Efficient Policy Learning for Robotic Navigation
Reinforcement learning methods, while effective for learning robotic navigation strategies, are known to be highly sample inefficient.
On the Possibilities of AI-Generated Text Detection
Our work focuses on the challenge of detecting outputs generated by Large Language Models (LLMs) to distinguish them from those generated by humans.
RE-MOVE: An Adaptive Policy Design for Robotic Navigation Tasks in Dynamic Environments via Language-Based Feedback
Reinforcement learning-based policies for continuous control robotic navigation tasks often fail to adapt to changes in the environment during real-time deployment, which may result in catastrophic failures.
STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning
Directed Exploration is a crucial challenge in reinforcement learning (RL), especially when rewards are sparse.
Model-based Reinforcement Learning
reinforcement-learning
+1
Beyond Exponentially Fast Mixing in Average-Reward Reinforcement Learning via Multi-Level Monte Carlo Actor-Critic
Many existing reinforcement learning (RL) methods employ stochastic gradient iteration on the back end, whose stability hinges upon a hypothesis that the data-generating process mixes exponentially fast with a rate parameter that appears in the step-size selection.
SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication
Furthermore, we provide theoretical results for IID and non-IID settings without any bounded-delay assumption for slow clients which is required by other asynchronous decentralized FL algorithms.
DC-MRTA: Decentralized Multi-Robot Task Allocation and Navigation in Complex Environments
At the higher level, we solve the task allocation by formulating it in terms of Markov Decision Processes and choosing the appropriate rewards to minimize the Total Travel Delay (TTD).
FedBC: Calibrating Global and Local Models via Federated Learning Beyond Consensus
In this work, we quantitatively calibrate the performance of global and local models in federated learning through a multi-criterion optimization-based framework, which we cast as a constrained program.
Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies
In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems.
Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithm
We propose a novel Conservative Natural Policy Gradient Primal-Dual Algorithm (C-NPG-PD) to achieve zero constraint violation while achieving state of the art convergence results for the objective value function.
Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning
Model-based approaches to reinforcement learning (MBRL) exhibit favorable performance in practice, but their theoretical guarantees in large spaces are mostly restricted to the setting when transition model is Gaussian or Lipschitz, and demands a posterior estimate whose representational complexity grows unbounded with time.
On the Hidden Biases of Policy Mirror Ascent in Continuous Action Spaces
Doing so incurs a persistent bias that appears in the attenuation rate of the expected policy gradient norm, which is inversely proportional to the radius of the action space.
Projection-Free Algorithm for Stochastic Bi-level Optimization
The proposed $\textbf{S}$tochastic $\textbf{C}$ompositional $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SCFW}$) is shown to achieve a sample complexity of $\mathcal{O}(\epsilon^{-2})$ for convex objectives and $\mathcal{O}(\epsilon^{-3})$ for non-convex objectives, at par with the state-of-the-art sample complexities for projection-free algorithms solving single-level problems.
Convergence Rates of Average-Reward Multi-agent Reinforcement Learning via Randomized Linear Programming
We establish that the sample complexity to obtain near-globally optimal solutions matches tight dependencies on the cardinality of the state and action spaces, and exhibits classical scalings with respect to the network in accordance with multi-agent optimization.
Multi-agent Reinforcement Learning
Reinforcement Learning (RL)
Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approach
To achieve that, we advocate the use of randomized primal-dual approach to solve the CMDP problems and propose a conservative stochastic primal-dual algorithm (CSPDA) which is shown to exhibit $\tilde{\mathcal{O}}\left(1/\epsilon^2\right)$ sample complexity to achieve $\epsilon$-optimal cumulative reward with zero constraint violations.
Wasserstein-Splitting Gaussian Process Regression for Heterogeneous Online Bayesian Inference
Gaussian processes (GPs) are a well-known nonparametric Bayesian inference technique, but they suffer from scalability problems for large sample sizes, and their performance can degrade for non-stationary or spatially heterogeneous data.
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control
To close this gap, we step towards persistent exploration in continuous space through policy parameterizations defined by distributions of heavier tails defined by tail-index parameter alpha, which increases the likelihood of jumping in state space.
MARL with General Utilities via Decentralized Shadow Reward Actor-Critic
DSAC augments the classic critic step by requiring agents to (i) estimate their local occupancy measure in order to (ii) estimate the derivative of the local utility with respect to their occupancy measure, i. e., the "shadow reward".
Conservative Stochastic Optimization with Expectation Constraints
In this work, we propose the FW-CSOA algorithm that is not only projection-free but also achieves zero constraint violation with $\O\left(T^{-\frac{1}{4}}\right)$ decay of the optimality gap.
Variational Policy Gradient Method for Reinforcement Learning with General Utilities
Analogously to the Policy Gradient Theorem \cite{sutton2000policy} available for RL with cumulative rewards, we derive a new Variational Policy Gradient Theorem for RL with general utilities, which establishes that the parametrized policy gradient may be obtained as the solution of a stochastic saddle point problem involving the Fenchel dual of the utility function.
Efficient Large-Scale Gaussian Process Bandits by Believing only Informative Actions
Experimentally, we observe state of the art accuracy and complexity tradeoffs for GP bandit algorithms on various hyper-parameter tuning tasks, suggesting the merits of managing the complexity of GPs in bandit settings
Regret and Belief Complexity Trade-off in Gaussian Process Bandits via Information Thresholding
Doing so permits us to precisely characterize the trade-off between regret bounds of GP bandit algorithms and complexity of the posterior distributions depending on the compression parameter $\epsilon$ for both discrete and continuous action sets.
Cautious Reinforcement Learning via Distributional Risk in the Dual Domain
To ameliorate this issue, we propose a new definition of risk, which we call caution, as a penalty function added to the dual objective of the linear programming (LP) formulation of reinforcement learning.
Optimally Compressed Nonparametric Online Learning
Batch training of machine learning models based on neural networks is now well established, whereas to date streaming methods are largely based on linear models.
Nonstationary Nonparametric Online Learning: Balancing Dynamic Regret and Model Parsimony
Prior works control dynamic regret growth only for linear models.
Adaptive Kernel Learning in Heterogeneous Networks
We consider learning in decentralized heterogeneous networks: agents seek to minimize a convex functional that aggregates data across the network, while only having access to their local data streams.
Online Learning over Dynamic Graphs via Distributed Proximal Gradient Algorithm
The empirical performance of the proposed algorithm is tested on the distributed dynamic sparse recovery problem, where it is shown to incur a dynamic regret that is close to that of the centralized algorithm.
