Search Results for author:
Found 59 papers, 13 papers with code
Combinatorial Pure Exploration for Dueling Bandit
For Borda winner, we establish a reduction of the problem to the original CPE-MAB setting and design PAC and exact algorithms that achieve both the sample complexity similar to that in the CPE-MAB setting (which is nearly optimal for a subclass of problems) and polynomial running time per round.
Offline-to-Online Multi-Agent Reinforcement Learning with Offline Value Function Memory and Sequential Exploration
Offline-to-Online Reinforcement Learning has emerged as a powerful paradigm, leveraging offline data for initialization and online fine-tuning to enhance both sample efficiency and performance.
uniINF: Best-of-Both-Worlds Algorithm for Parameter-Free Heavy-Tailed MABs
To our knowledge, uniINF is the first parameter-free algorithm to achieve the BoBW property for the heavy-tailed MAB problem.
Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks
In general, GFlowNets are trained by fitting the forward flow to the backward flow on sampled training objects.
Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training
However, a direct adoption of DST fails to yield satisfactory MARL agents, leading to breakdowns in value learning within deep sparse value-based MARL models.
Adversarial Network Optimization under Bandit Feedback: Maximizing Utility in Non-Stationary Multi-Hop Networks
Stochastic Network Optimization (SNO) concerns scheduling in stochastic queueing systems.
Mixed Sparsity Training: Achieving 4$\times$ FLOP Reduction for Transformer Pretraining
Throughout these phases, the model is trained with a dynamically evolving sparse topology and an HSA mechanism to maintain performance and minimize training FLOPs concurrently.
RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learning
Effective action abstraction is crucial in tackling challenges associated with large action spaces in Imperfect Information Extensive-Form Games (IIEFGs).
Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation
This work pioneers regret analysis of risk-sensitive reinforcement learning in partially observable environments with hindsight observation, addressing a gap in theoretical exploration.
Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximation
In this paper, we introduce a general framework on Risk-Sensitive Distributional Reinforcement Learning (RS-DisRL), with static Lipschitz Risk Measures (LRM) and general function approximation.
Distributional Reinforcement Learning
reinforcement-learning
+2
LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
Latent Consistency Models (LCMs) have achieved impressive performance in accelerating text-to-image generative tasks, producing high-quality images with minimal inference steps.
A Quadratic Synchronization Rule for Distributed Deep Learning
In distributed deep learning with data parallelism, synchronizing gradients at each training step can cause a huge communication overhead, especially when many nodes work together to train large models.
One is More: Diverse Perspectives within a Single Network for Efficient DRL
Deep reinforcement learning has achieved remarkable performance in various domains by leveraging deep neural networks for approximating value functions and policies.
Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
Inspired by Consistency Models (song et al.), we propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion (rombach et al).
Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedback
Risk-sensitive reinforcement learning (RL) aims to optimize policies that balance the expected reward and risk.
Beyond Conservatism: Diffusion Policies in Offline Multi-agent Reinforcement Learning
We present a novel Diffusion Offline Multi-agent Model (DOM2) for offline Multi-Agent Reinforcement Learning (MARL).
RePreM: Representation Pre-training with Masked Model for Reinforcement Learning
Inspired by the recent success of sequence modeling in RL and the use of masked language model for pre-training, we propose a masked model for pre-training in RL, RePreM (Representation Pre-training with Masked Model), which trains the encoder combined with transformer blocks to predict the masked states or actions in a trajectory.
Queue Scheduling with Adversarial Bandit Learning
In this paper, we study scheduling of a queueing system with zero knowledge of instantaneous network conditions.
Why (and When) does Local SGD Generalize Better than SGD?
Local SGD is a communication-efficient variant of SGD for large-scale training, where multiple GPUs perform SGD independently and average the model parameters periodically.
Stochastic Generative Flow Networks
Generative Flow Networks (or GFlowNets for short) are a family of probabilistic agents that learn to sample complex combinatorial structures through the lens of "inference as control".
Multi-task Representation Learning for Pure Exploration in Linear Bandits
In these two problems, all tasks share a common low-dimensional linear representation, and our goal is to leverage this feature to accelerate the best arm (policy) identification process for all tasks.
Banker Online Mirror Descent: A Universal Approach for Delayed Online Bandit Learning
\texttt{Banker-OMD} leads to the first delayed scale-free adversarial MAB algorithm achieving $\widetilde{\mathcal O}(\sqrt{K}L(\sqrt T+\sqrt D))$ regret and the first delayed adversarial linear bandit algorithm achieving $\widetilde{\mathcal O}(\text{poly}(n)(\sqrt{T} + \sqrt{D}))$ regret.
Dueling Bandits: From Two-dueling to Multi-dueling
DoublerBAI provides a generic schema for translating known results on best arm identification algorithms to the dueling bandit problem, and achieves a regret bound of $O(\ln T)$.
Generative Augmented Flow Networks
We specify intermediate rewards by intrinsic motivation to tackle the exploration problem in sparse reward environments.
Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learning
Multi-user delay constrained scheduling is important in many real-world applications including wireless communication, live streaming, and cloud computing.
Nearly Minimax Optimal Reinforcement Learning with Linear Function Approximation
We study reinforcement learning with linear function approximation where the transition probability and reward functions are linear with respect to a feature mapping $\boldsymbol{\phi}(s, a)$.
Provable Generalization of Overparameterized Meta-learning Trained with SGD
Despite the superior empirical success of deep meta-learning, theoretical understanding of overparameterized meta-learning is still limited.
Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Path
For Worst Path RL, we propose an efficient algorithm with constant upper and lower bounds.
RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch
Training deep reinforcement learning (DRL) models usually requires high computation costs.
Network Topology Optimization via Deep Reinforcement Learning
A2C-GS consists of three novel components, including a verifier to validate the correctness of a generated network topology, a graph neural network (GNN) to efficiently approximate topology rating, and a DRL actor layer to conduct a topology search.
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Recently, it has been observed that the best uni-modal network outperforms the jointly trained multi-modal network, which is counter-intuitive since multiple signals generally bring more information.
Adaptive Best-of-Both-Worlds Algorithm for Heavy-Tailed Multi-Armed Bandits
Specifically, we design an algorithm \texttt{HTINF}, when the heavy-tail parameters $\alpha$ and $\sigma$ are known to the agent, \texttt{HTINF} simultaneously achieves the optimal regret for both stochastic and adversarial environments, without knowing the actual environment type a-priori.
Regularized Softmax Deep Multi-Agent Q-Learning
Tackling overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning, but has received comparatively little attention in the multi-agent setting.
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
Conservatism has led to significant progress in offline reinforcement learning (RL) where an agent learns from pre-collected datasets.
Simultaneously Achieving Sublinear Regret and Constraint Violations for Online Convex Optimization with Time-varying Constraints
In this paper, we develop a novel virtual-queue-based online algorithm for online convex optimization (OCO) problems with long-term and time-varying constraints and conduct a performance analysis with respect to the dynamic regret and constraint violations.
Collaborative Pure Exploration in Kernel Bandit
In this paper, we formulate a Collaborative Pure Exploration in Kernel Bandit problem (CoPE-KB), which provides a novel model for multi-agent multi-task decision making under limited communication and general reward functions, and is applicable to many online learning tasks, e. g., recommendation systems and network scheduling.
Scale-Free Adversarial Multi-Armed Bandit with Arbitrary Feedback Delays
We consider the Scale-Free Adversarial Multi-Armed Bandit (MAB) problem with unrestricted feedback delays.
Banker Online Mirror Descent
In particular, it leads to the first delayed adversarial linear bandit algorithm achieving $\tilde{O}(\text{poly}(n)(\sqrt{T} + \sqrt{D}))$ regret.
What Makes Multi-modal Learning Better than Single (Provably)
The world provides us with data of multiple modalities.
The best of both worlds: stochastic and adversarial episodic MDPs with unknown transition
We consider the best-of-both-worlds problem for learning an episodic Markov Decision Process through $T$ episodes, with the goal of achieving $\widetilde{\mathcal{O}}(\sqrt{T})$ regret when the losses are adversarial and simultaneously $\mathcal{O}(\text{polylog}(T))$ regret when the losses are (almost) stochastic.
Fast Federated Learning in the Presence of Arbitrary Device Unavailability
In this case, the convergence of popular FL algorithms such as FedAvg is severely influenced by the straggling devices.
Regularized Softmax Deep Multi-Agent $Q$-Learning
Tackling overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning, but has received comparatively little attention in the multi-agent setting.
Continuous Mean-Covariance Bandits
To the best of our knowledge, this is the first work that considers option correlation in risk-aware bandits and explicitly quantifies how arbitrary covariance structures impact the learning performance.
A One-Size-Fits-All Solution to Conservative Bandit Problems
In this paper, we study a family of conservative bandit problems (CBPs) with sample-path reward constraints, i. e., the learner's reward performance must be at least as well as a given baseline at any time.
Adaptive Algorithms for Multi-armed Bandit with Composite and Anonymous Feedback
Existing results on this model require prior knowledge about the reward interval size as an input to their algorithms.
Restless-UCB, an Efficient and Low-complexity Algorithm for Online Restless Bandits
Compared to existing algorithms, our result eliminates the exponential factor (in $M, N$) in the regret upper bound, due to a novel exploitation of the sparsity in transitions in general restless bandit problems.
Softmax Deep Double Deterministic Policy Gradients
A widely-used actor-critic reinforcement learning algorithm for continuous control, Deep Deterministic Policy Gradients (DDPG), suffers from the overestimation problem, which can negatively affect the performance.
Combinatorial Pure Exploration of Dueling Bandit
For Borda winner, we establish a reduction of the problem to the original CPE-MAB setting and design PAC and exact algorithms that achieve both the sample complexity similar to that in the CPE-MAB setting (which is nearly optimal for a subclass of problems) and polynomial running time per round.
Multi-Agent Reinforcement Learning in Stochastic Networked Systems
We study multi-agent reinforcement learning (MARL) in a stochastic network of agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Exploration by Maximizing Rényi Entropy for Reward-Free RL Framework
In the planning phase, the agent computes a good policy for any reward function based on the dataset without further interacting with the environment.
Multi-Path Policy Optimization
Recent years have witnessed a tremendous improvement of deep reinforcement learning.
Reinforcement Learning with Dynamic Boltzmann Softmax Updates
In this paper, we propose to update the value function with dynamic Boltzmann softmax (DBS) operator, which has good convergence property in the setting of planning and learning.
Multi-armed Bandits with Compensation
We propose and study the known-compensation multi-arm bandit (KCMAB) problem, where a system controller offers a set of arms to many short-term players for $T$ steps.
Double Quantization for Communication-Efficient Distributed Optimization
In this paper, to reduce the communication complexity, we propose \emph{double quantization}, a general scheme for quantizing both model parameters and gradients.
Beyond the Click-Through Rate: Web Link Selection with Multi-level Feedback
The web link selection problem is to select a small subset of web links from a large web link pool, and to place the selected links on a web page that can only accommodate a limited number of links, e. g., advertisements, recommendations, or news feeds.
A Deep Reinforcement Learning Framework for Rebalancing Dockless Bike Sharing Systems
Different from existing methods that often ignore spatial information and rely heavily on accurate prediction, HRP captures both spatial and temporal dependencies using a divide-and-conquer structure with an embedded localized module.
Multi-level Feedback Web Links Selection Problem: Learning and Optimization
To our best knowledge, we are the first to model the links selection problem as a constrained multi-armed bandit problem and design an effective links selection algorithm by learning the links' multi-level structure with provable \emph{sub-linear} regret and violation bounds.
Fast Stochastic Variance Reduced ADMM for Stochastic Composition Optimization
We consider the stochastic composition optimization problem proposed in \cite{wang2017stochastic}, which has applications ranging from estimation to statistical and machine learning.
The Power of Online Learning in Stochastic Network Optimization
We prove strong performance guarantees of the proposed algorithms: $\mathtt{OLAC}$ and $\mathtt{OLAC2}$ achieve the near-optimal $[O(\epsilon), O([\log(1/\epsilon)]^2)]$ utility-delay tradeoff and $\mathtt{OLAC2}$ possesses an $O(\epsilon^{-2/3})$ convergence time.
