Search Results for author:
Found 105 papers, 45 papers with code
Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
Many artificial intelligence (AI) applications often require multiple intelligent agents to work in a collaborative effort.
Adversarial Variational Bayes Methods for Tweedie Compound Poisson Mixed Models
Mixed models with random effects account for the covariance structure related to the grouping hierarchy in the data.
A Study of AI Population Dynamics with Million-agent Reinforcement Learning
We conduct an empirical study on discovering the ordered collective dynamics obtained by a population of intelligence agents, driven by million-agent reinforcement learning.
Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
We propose a new sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large datasets and multimodal distributions.
Mean Field Multi-Agent Reinforcement Learning
Existing multi-agent reinforcement learning methods are limited typically to a small number of agents.
Factorized Q-Learning for Large-Scale Multi-Agent Systems
Deep Q-learning has achieved a significant success in single-agent decision making tasks.
Multiagent Systems
Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series
As an indicator of the level of risk or the degree of variation, volatility is important to analyse the financial market, and it is taken into consideration in various decision-making processes in financial activities.
Can Deep Learning Predict Risky Retail Investors? A Case Study in Financial Risk Behavior Forecasting
The results of employing a deep network for operational risk forecasting confirm the feature learning capability of deep learning, provide guidance on designing a suitable network architecture and demonstrate the superiority of deep learning over machine learning and rule-based benchmarks.
Modelling Bounded Rationality in Multi-Agent Interactions by Generalized Recursive Reasoning
Though limited in real-world decision making, most multi-agent reinforcement learning (MARL) models assume perfectly rational agents -- a property hardly met due to individual's cognitive limitation and/or the tractability of the decision problem.
Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning
Our methods are tested on both the matrix game and the differential game, which have a non-trivial equilibrium where common gradient-based methods fail to converge.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Disentangling Dynamics and Returns: Value Function Decomposition with Future Prediction
Value functions are crucial for model-free Reinforcement Learning (RL) to obtain a policy implicitly or guide the policy updates.
Replica-exchange Nosé-Hoover dynamics for Bayesian learning on large datasets
In this paper, we present a new practical method for Bayesian learning that can rapidly draw representative samples from complex posterior distributions with multiple isolated modes in the presence of mini-batch noise.
Spectral-based Graph Convolutional Network for Directed Graphs
Our approach can work directly on directed graph data in semi-supervised nodes classification tasks.
Bi-level Actor-Critic for Multi-agent Coordination
Coordination is one of the essential problems in multi-agent systems.
Multiagent Systems
$α^α$-Rank: Practically Scaling $α$-Rank through Stochastic Optimisation
Furthermore, we also show successful results on large joint strategy profiles with a maximum size in the order of $\mathcal{O}(2^{25})$ ($\approx 33$ million joint strategies) -- a setting not evaluable using $\alpha$-Rank with reasonable computational budget.
Multi-Agent Determinantal Q-Learning
Though practical, current methods rely on restrictive assumptions to decompose the centralized value function across agents for execution.
Learning to Infer User Hidden States for Online Sequential Advertising
To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important.
What About Taking Policy as Input of Value Function: Policy-extended Value Function Approximator
The value function lies in the heart of Reinforcement Learning (RL), which defines the long-term evaluation of a policy in a given state.
What About Inputing Policy in Value Function: Policy Representation and Policy-extended Value Function Approximator
We study Policy-extended Value Function Approximator (PeVFA) in Reinforcement Learning (RL), which extends conventional value function approximator (VFA) to take as input not only the state (and action) but also an explicit policy representation.
SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving.
An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
In this work, we provide a monograph on MARL that covers both the fundamentals and the latest developments in the research frontier.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Replica-Exchange Nos\'e-Hoover Dynamics for Bayesian Learning on Large Datasets
In this paper, we present a new practical method for Bayesian learning that can rapidly draw representative samples from complex posterior distributions with multiple isolated modes in the presence of mini-batch noise.
Multi-Agent Trust Region Learning
We derive the lower bound of agents' payoff improvements for MATRL methods, and also prove the convergence of our method on the meta-game fixed points.
Robust Multi-Agent Reinforcement Learning Driven by Correlated Equilibrium
Therefore, to achieve robust CMARL, we introduce novel strategies to encourage agents to learn correlated equilibrium while maximally preserving the convenience of the decentralized execution.
Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
Multiagent reinforcement learning (MARL) has achieved a remarkable amount of success in solving various types of video games.
Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction
Value function is the central notion of Reinforcement Learning (RL).
Online Double Oracle
Solving strategic games with huge action space is a critical yet under-explored topic in economics, operations research and artificial intelligence.
Modelling Behavioural Diversity for Learning in Open-Ended Games
Promoting behavioural diversity is critical for solving games with non-transitive dynamics where strategic cycles exist, and there is no consistent winner (e. g., Rock-Paper-Scissors).
Learning to Shape Rewards using a Game of Two Partners
Reward shaping (RS) is a powerful method in reinforcement learning (RL) for overcoming the problem of sparse or uninformative rewards.
Learning to Safely Exploit a Non-Stationary Opponent
On the other hand, overfitting to an opponent (i. e., exploiting only one specific type of opponent) makes the learning player easily exploitable by others.
Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment
In addition, we use a novel agent network named Population Invariant agent with Transformer (PIT) to realize the coordination transfer in more varieties of scenarios.
Neural Auto-Curricula
When solving two-player zero-sum games, multi-agent reinforcement learning (MARL) algorithms often create populations of agents where, at each iteration, a new agent is discovered as the best response to a mixture over the opponent population.
MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning
Our framework is comprised of three key components: (1) a centralized task dispatching model, which supports the self-generated tasks and scalable training with heterogeneous policy combinations; (2) a programming architecture named Actor-Evaluator-Learner, which achieves high parallelism for both training and sampling, and meets the evaluation requirement of auto-curriculum learning; (3) a higher-level abstraction of MARL training paradigms, which enables efficient code reuse and flexible deployments on different distributed computing paradigms.
Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Games
With this unified diversity measure, we design the corresponding diversity-promoting objective and population effectivity when seeking the best responses in open-ended learning.
A Game-Theoretic Approach to Multi-Agent Trust Region Optimization
Trust region methods are widely applied in single-agent reinforcement learning problems due to their monotonic performance-improvement guarantee at every iteration.
Is Nash Equilibrium Approximator Learnable?
In this paper, we investigate the learnability of the function approximator that approximates Nash equilibrium (NE) for games generated from a distribution.
Settling the Variance of Multi-Agent Policy Gradients
In multi-agent RL (MARL), although the PG theorem can be naturally extended, the effectiveness of multi-agent PG (MAPG) methods degrades as the variance of gradient estimates increases rapidly with the number of agents.
On the Complexity of Computing Markov Perfect Equilibrium in General-Sum Stochastic Games
Similar to the role of Markov decision processes in reinforcement learning, Stochastic Games (SGs) lay the foundation for the study of multi-agent reinforcement learning (MARL) and sequential agent interactions.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Revisiting the Characteristics of Stochastic Gradient Noise and Dynamics
In this paper, we characterize the noise of stochastic gradients and analyze the noise-induced dynamics during training deep neural networks by gradient-based optimizers.
Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning
In this paper, we extend the theory of trust region learning to MARL.
Offline Pre-trained Multi-Agent Decision Transformer
Offline reinforcement learning leverages static datasets to learn optimal policies with no necessity to access the environment.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Multi-Agent Constrained Policy Optimisation
To fill these gaps, in this work, we formulate the safe MARL problem as a constrained Markov game and solve it with policy optimisation methods.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Online Markov Decision Processes with Non-oblivious Strategic Adversary
In this setting, we first demonstrate that MDP-Expert, an existing algorithm that works well with oblivious adversaries can still apply and achieve a policy regret bound of $\mathcal{O}(\sqrt{T \log(L)}+\tau^2\sqrt{ T \log(|A|)})$ where $L$ is the size of adversary's pure strategy set and $|A|$ denotes the size of agent's action space.
Measuring the Non-Transitivity in Chess
In this paper, we quantify the non-transitivity in Chess through real-world data from human players.
DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Intervention
In this paper, we introduce a new generation of RL solvers that learn to minimise safety violations while maximising the task reward to the extent that can be tolerated by the safe policy.
A Game-Theoretic Approach for Improving Generalization Ability of TSP Solvers
In this paper, we introduce a two-player zero-sum framework between a trainable \emph{Solver} and a \emph{Data Generator} to improve the generalization ability of deep learning-based solvers for Traveling Salesman Problem (TSP).
Towards Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Games
With this unified diversity measure, we design the corresponding diversity-promoting objective and population effectivity when seeking the best responses in open-ended learning.
Neural Auto-Curricula in Two-Player Zero-Sum Games
When solving two-player zero-sum games, multi-agent reinforcement learning (MARL) algorithms often create populations of agents where, at each iteration, a new agent is discovered as the best response to a mixture over the opponent population.
Multi-agent Reinforcement Learning
Vocal Bursts Valence Prediction
Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks
In this paper, we facilitate the research by providing large-scale datasets, and use them to examine the usage of the Decision Transformer in the context of MARL.
A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning
Gradient-based Meta-RL (GMRL) refers to methods that maintain two-level optimisation procedures wherein the outer-loop meta-learner guides the inner-loop gradient-based reinforcement learner to achieve fast adaptations.
Efficient Policy Space Response Oracles
Policy Space Response Oracle methods (PSRO) provide a general solution to learn Nash equilibrium in two-player zero-sum games but suffer from two drawbacks: (1) the computation inefficiency due to the need for consistent meta-game evaluation via simulations, and (2) the exploration inefficiency due to finding the best response against a fixed meta-strategy at every epoch.
Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learning
Value function decomposition is becoming a popular rule of thumb for scaling up multi-agent reinforcement learning (MARL) in cooperative games.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Settling the Communication Complexity for Distributed Offline Reinforcement Learning
We study a novel setting in offline reinforcement learning (RL) where a number of distributed machines jointly cooperate to solve the problem but only one single round of communication is allowed and there is a budget constraint on the total number of information (in terms of bits) that each machine can send out.
Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Framework
To break this curse, we propose a unified agent permutation framework that exploits the permutation invariance (PI) and permutation equivariance (PE) inductive biases to reduce the multiagent state space.
On the Convergence of Fictitious Play: A Decomposition Approach
Fictitious play (FP) is one of the most fundamental game-theoretical learning frameworks for computing Nash equilibrium in $n$-player games, which builds the foundation for modern multi-agent learning algorithms.
A Review of Safe Reinforcement Learning: Methods, Theory and Applications
To establish a good foundation for future research in this thread, in this paper, we provide a review for safe RL from the perspectives of methods, theory and applications.
Multi-Agent Reinforcement Learning is a Sequence Modeling Problem
In this paper, we introduce a novel architecture named Multi-Agent Transformer (MAT) that effectively casts cooperative multi-agent reinforcement learning (MARL) into SM problems wherein the task is to map agents' observation sequence to agents' optimal action sequence.
A Game-Theoretic Framework for Managing Risk in Multi-Agent Systems
Although there are equilibrium concepts in game theory that take into account risk aversion, they either assume that agents are risk-neutral with respect to the uncertainty caused by the actions of other agents, or they are not guaranteed to exist.
Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
In this study, we propose the Bimanual Dexterous Hands Benchmark (Bi-DexHands), a simulator that involves two dexterous hands with tens of bimanual manipulation tasks and thousands of target objects.
Scalable Model-based Policy Optimization for Decentralized Networked Systems
Reinforcement learning algorithms require a large amount of samples; this often limits their real-world applications on even simple tasks.
Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARL
The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in the artificial intelligence (AI) research community.
Debias the Black-box: A Fair Ranking Framework via Knowledge Distillation
Deep neural networks can capture the intricate interaction history information between queries and documents, because of their many complicated nonlinear units, allowing them to provide correct search recommendations.
Constrained Update Projection Approach to Safe Policy Optimization
Compared to previous safe RL methods, CUP enjoys the benefits of 1) CUP generalizes the surrogate functions to generalized advantage estimator (GAE), leading to strong empirical performance.
End-to-End Affordance Learning for Robotic Manipulation
Such contact prediction process then leads to an end-to-end affordance learning framework that can generalize over different types of manipulation tasks.
MSRL: Distributed Reinforcement Learning with Dataflow Fragments
Yet, current distributed RL systems tie the definition of RL algorithms to their distributed execution: they hard-code particular distribution strategies and only accelerate specific parts of the computation (e. g. policy network updates) on GPU workers.
GenDexGrasp: Generalizable Dexterous Grasping
By leveraging the contact map as a hand-agnostic intermediate representation, GenDexGrasp efficiently generates diverse and plausible grasping poses with a high success rate and can transfer among diverse multi-fingered robotic hands.
MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
A significant challenge facing researchers in the area of multi-agent reinforcement learning (MARL) pertains to the identification of a library that can offer fast and compatible development for multi-agent tasks and algorithm combinations, while obviating the need to consider compatibility issues.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
TorchOpt: An Efficient Library for Differentiable Optimization
TorchOpt further provides a high-performance distributed execution runtime.
Contextual Transformer for Offline Meta Reinforcement Learning
Firstly, we propose prompt tuning for offline RL, where a context vector sequence is concatenated with the input to guide the conditional policy generation.
ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency
In the learning phase, each agent minimizes the TD error that is dependent on how the subsequent agents have reacted to their chosen action.
Ranked #1 on
SMAC
on SMAC 3s5z_vs_3s6z
ASP: Learn a Universal Neural Solver!
Applying machine learning to combinatorial optimization problems has the potential to improve both efficiency and accuracy.
DPPMask: Masked Image Modeling with Determinantal Point Processes
In particular, we evaluate our method on two representative MIM frameworks, MAE and iBOT.
UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning
We propose a novel, object-agnostic method for learning a universal policy for dexterous object grasping from realistic point cloud observations and proprioceptive information under a table-top setting, namely UniDexGrasp++.
STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning
It first decomposes the global return back to each time step, then utilizes the Shapley Value to redistribute the individual payoff from the decomposed global reward.
Heterogeneous-Agent Reinforcement Learning
The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in AI research.
OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research
AI systems empowered by reinforcement learning (RL) algorithms harbor the immense potential to catalyze societal advancement, yet their deployment is often impeded by significant safety concerns.
Heterogeneous Value Alignment Evaluation for Large Language Models
We conduct evaluations with new auto-metric \textit{value rationality} to represent the ability of LLMs to align with specific values.
Deep Reinforcement Learning with Task-Adaptive Retrieval via Hypernetwork
Deep reinforcement learning algorithms are usually impeded by sampling inefficiency, heavily depending on multiple interactions with the environment to acquire accurate decision-making capabilities.
Maximum Entropy Heterogeneous-Agent Reinforcement Learning
We embed cooperative MARL problems into probabilistic graphical models, from which we derive the maximum entropy (MaxEnt) objective for MARL.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Large Sequence Models for Sequential Decision-Making: A Survey
Transformer architectures have facilitated the development of large-scale and general-purpose sequence models for prediction tasks in natural language processing and computer vision, e. g., GPT-3 and Swin Transformer.
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs).
SafeDreamer: Safe Reinforcement Learning with World Models
Existing Safe Reinforcement Learning (SafeRL) methods, which rely on cost functions to enforce safety, often fail to achieve zero-cost performance in complex scenarios, especially vision-only tasks.
Theoretically Guaranteed Policy Improvement Distilled from Model-Based Planning
Model-based reinforcement learning (RL) has demonstrated remarkable successes on a range of continuous control tasks due to its high sample efficiency.
JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Games
This paper presents an empirical exploration of non-transitivity in perfect-information games, specifically focusing on Xiangqi, a traditional Chinese board game comparable in game-tree complexity to chess and shogi.
ProAgent: Building Proactive Cooperative Agents with Large Language Models
Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems.
Mixup-Augmented Meta-Learning for Sample-Efficient Fine-Tuning of Protein Simulators
Molecular dynamics simulations have emerged as a fundamental instrument for studying biomolecules.
Measuring Value Understanding in Language Models through Discriminator-Critique Gap
We argue that truly understanding values in LLMs requires considering both "know what" and "know why".
Red Teaming Game: A Game-Theoretic Framework for Red Teaming Language Models
In this paper, we present Red-teaming Game (RTG), a general game-theoretic framework without manual annotation.
GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models
This paper introduces a distributed, GPU-centric experience replay system, GEAR, designed to perform scalable reinforcement learning (RL) with large sequence models (such as transformers).
MIR2: Towards Provably Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization
Existing max-min optimization techniques in robust MARL seek to enhance resilience by training agents against worst-case adversaries, but this becomes intractable as the number of agents grows, leading to exponentially increasing worst-case scenarios.
MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning
Building a single generalist agent with strong zero-shot capability has recently sparked significant advancements.
Safe RLHF: Safe Reinforcement Learning from Human Feedback
However, the inherent tension between the objectives of helpfulness and harmlessness presents a significant challenge during LLM training.
Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark
By introducing this benchmark, we aim to facilitate the evaluation and comparison of safety performance, thus fostering the development of reinforcement learning for safer, more reliable, and responsible real-world applications.
Grasp Multiple Objects with One Hand
The intricate kinematics of the human hand enable simultaneous grasping and manipulation of multiple objects, essential for tasks such as object transfer and in-hand manipulation.
AI Alignment: A Comprehensive Survey
The former aims to make AI systems aligned via alignment training, while the latter aims to gain evidence about the systems' alignment and govern them appropriately to avoid exacerbating misalignment risks.
JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models
Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents.
A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
In this paper, from a novel perspective, we systematically study the challenges that remain in O2O RL and identify that the reason behind the slow improvement of the performance and the instability of online finetuning lies in the inaccurate Q-value estimation inherited from offline pretraining.
CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents
Within CivRealm, we provide interfaces for two typical agent types: tensor-based agents that focus on learning, and language-based agents that emphasize reasoning.
Panacea: Pareto Alignment via Preference Adaptation for LLMs
Our work marks a step forward in effectively and efficiently aligning models to diverse and intricate human preferences in a controllable and Pareto-optimal manner.
Aligner: Achieving Efficient Alignment through Weak-to-Strong Correction
Here we introduce Aligner, a new efficient alignment paradigm that bypasses the whole RLHF process by learning the correctional residuals between the aligned and the unaligned answers.
Rethinking Information Structures in RLHF: Reward Generalization from a Graph Theory Perspective
Then, based on this framework, we introduce the IBN to analyze generalization in the reward modeling stage of RLHF.
Incentive Compatibility for AI Alignment in Sociotechnical Systems: Positions and Prospects
The burgeoning integration of artificial intelligence (AI) into human society brings forth significant implications for societal governance and safety.
INSIGHT: End-to-End Neuro-Symbolic Visual Reinforcement Learning with Language Explanations
In this paper, we present a framework that is capable of learning structured states and symbolic policies simultaneously, whose key idea is to overcome the efficiency bottleneck by distilling vision foundation models into a scalable perception module.
AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents
Traditional approaches in physics-based motion generation, centered around imitation learning and reward shaping, often struggle to adapt to new scenarios.
