Search Results for author:
Found 28 papers, 9 papers with code
ViMo: A Generative Visual GUI World Model for App Agents
We propose a novel data representation, the Symbolic Text Representation~(STR) to overlay text content with symbolic placeholders while preserving graphics.
ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoning
Evaluating large language models (LLMs) poses significant challenges, particularly due to issues of data contamination and the leakage of correct answers.
Advancing Autonomous VLM Agents via Variational Subgoal-Conditioned Reinforcement Learning
State-of-the-art (SOTA) reinforcement learning (RL) methods have enabled vision-language model (VLM) agents to learn from interaction with online environments without human supervision.
AppVLM: A Lightweight Vision Language Model for Online App Control
In this work, we introduce AppVLM, a lightweight Vision-Language Model (VLM).
GUI Agents with Foundation Models: A Comprehensive Survey
Recent advances in foundation models, particularly Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs), have facilitated the development of intelligent agents capable of performing complex tasks.
Large Language Models Orchestrating Structured Reasoning Achieve Kaggle Grandmaster Level
We introduce Agent K v1. 0, an end-to-end autonomous data science agent designed to automate, optimise, and generalise across diverse data science tasks.
Lightweight Neural App Control
This paper introduces a novel mobile phone control architecture, termed ``app agents", for efficient interactions and controls across various Android apps.
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
SPA-Bench offers three key contributions: (1) A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines; (2) A plug-and-play framework enabling real-time agent interaction with Android devices, integrating over ten agents with the flexibility to add more; (3) A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption.
DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents
This paper introduces DistRL, a novel framework designed to enhance the efficiency of online RL fine-tuning for mobile device control agents.
Learning Precise Affordances from Egocentric Videos for Robotic Manipulation
In this paper, we present a streamlined affordance learning system that encompasses data collection, effective model training, and robot deployment.
ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning
Key features of the framework include: integration of ROS with an AI agent connected to a plethora of open-source and commercial LLMs, automatic extraction of a behavior from the LLM output and execution of ROS actions/services, support for three behavior modes (sequence, behavior tree, state machine), imitation learning for adding new robot actions to the library of possible actions, and LLM reflection via human and environment feedback.
Vim-F: Visual State Space Model Benefiting from Learning in the Frequency Domain
In recent years, State Space Models (SSMs) with efficient hardware-aware designs, known as the Mamba deep learning models, have made significant progress in modeling long sequences such as language understanding.
SFANet: Spatial-Frequency Attention Network for Weather Forecasting
Weather forecasting plays a critical role in various sectors, driving decision-making and risk management.
Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control
Learning a universal policy across different robot morphologies can significantly improve learning efficiency and enable zero-shot generalization to unseen morphologies.
Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning
This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies.
A survey on algorithms for Nash equilibria in finite normal-form games
This paper reviews various algorithms computing the Nash equilibrium and its approximation solutions in finite normal-form games from both theoretical and empirical perspectives.
ChessGPT: Bridging Policy Learning and Language Modeling
Thus, we propose ChessGPT, a GPT model bridging policy learning and language modeling by integrating data from these two sources in Chess games.
Traj-MAE: Masked Autoencoders for Trajectory Prediction
Specifically, our Traj-MAE employs diverse masking strategies to pre-train the trajectory encoder and map encoder, allowing for the capture of social and temporal information among agents while leveraging the effect of environment from multiple granularities.
Taming Multi-Agent Reinforcement Learning with Estimator Variance Reduction
Centralised training with decentralised execution (CT-DE) serves as the foundation of many leading multi-agent reinforcement learning (MARL) algorithms.
Timing is Everything: Learning to Act Selectively with Costly Actions and Budgetary Constraints
In this paper, we introduce a reinforcement learning (RL) framework named \textbf{L}earnable \textbf{I}mpulse \textbf{C}ontrol \textbf{R}einforcement \textbf{A}lgorithm (LICRA), for learning to optimally select both when to act and which actions to take when actions incur costs.
Learning Explicit Credit Assignment for Multi-agent Joint Q-learning
In contrast, we formulate an \emph{explicit} credit assignment problem where each agent gives its suggestion about how to weight individual Q-values to explicitly maximize the joint Q-value, besides guaranteeing the Bellman optimality of the joint Q-value.
Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment
In addition, we use a novel agent network named Population Invariant agent with Transformer (PIT) to realize the coordination transfer in more varieties of scenarios.
Robust Multi-Agent Reinforcement Learning Driven by Correlated Equilibrium
Therefore, to achieve robust CMARL, we introduce novel strategies to encourage agents to learn correlated equilibrium while maximally preserving the convenience of the decentralized execution.
Interpretable Reinforcement Learning With Neural Symbolic Logic
Recent progress in deep reinforcement learning (DRL) can be largely attributed to the use of neural networks.
Deep Reinforcement Learning
Hierarchical Reinforcement Learning
+3
SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving.
Multi-Agent Determinantal Q-Learning
Though practical, current methods rely on restrictive assumptions to decompose the centralized value function across agents for execution.
A Survey of Deep Reinforcement Learning in Video Games
In this paper, we survey the progress of DRL methods, including value-based, policy gradient, and model-based algorithms, and compare their main techniques and properties.
StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning
With reinforcement learning and curriculum transfer learning, our units are able to learn appropriate strategies in StarCraft micromanagement scenarios.
