Search Results for author:
Found 39 papers, 11 papers with code
Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation
Egocentric gaze anticipation serves as a key building block for the emerging capability of Augmented Reality.
Learning in Factored Domains with Information-Constrained Visual Representations
This raises the question of how humans learn to efficiently represent visual information in a manner useful for learning tasks.
Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks
Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and storage costs.
Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games
We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes.
Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria
By directly comparing active equilibria to Nash equilibria in these examples, we find that active equilibria find more effective solutions than Nash equilibria, concluding that an active equilibrium is the desired solution for multiagent learning settings.
Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and accuracy; or, in multi-task learning where multiple tasks are optimized jointly, sharing inductive bias between them.
In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation
To this end, we design the transformer encoder to embed the global context as one additional visual token and further propose a novel Global-Local Correlation (GLC) module to explicitly model the correlation of the global token and each local token.
Generative Adversarial Network for Future Hand Segmentation from Egocentric Video
We introduce the novel problem of anticipating a time series of future hand masks from egocentric video.
Influencing Long-Term Behavior in Multiagent Reinforcement Learning
An effective approach that has recently emerged for addressing this non-stationarity is for each agent to anticipate the learning of other agents and influence the evolution of future policies towards desirable behavior for its own benefit.
Hierarchical Reinforcement Learning with AI Planning Models
Our approach defines options in hierarchical reinforcement learning (HRL) from AIP operators by establishing a correspondence between the state transition model of AI planning problem and the abstract state transition system of a Markov Decision Process (MDP).
Local Explanations for Reinforcement Learning
Many works in explainable AI have focused on explaining black-box classification models.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Learning Multi-Objective Curricula for Robotic Policy Learning
Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL).
Interpreting Reinforcement Policies through Local Behaviors
Many works in explainable AI have focused on explaining black-box classification models.
Context-Specific Representation Abstraction for Deep Option Learning
Hierarchical reinforcement learning has focused on discovering temporally extended actions, such as options, that can provide benefits in problems requiring extensive exploration.
Egocentric Activity Recognition and Localization on a 3D Map
Given a video captured from a first person perspective and the environment context of where the video is recorded, can we recognize what the person is doing and identify where the action occurs in the 3D space?
4D Human Body Capture from Egocentric Video via 3D Scene Grounding
We introduce a novel task of reconstructing a time series of second-person 3D human body meshes from monocular egocentric videos.
Consolidation via Policy Information Regularization in Deep RL for Multi-Agent Games
This paper introduces an information-theoretic constraint on learned policy complexity in the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) reinforcement learning algorithm.
A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning
A fundamental challenge in multiagent reinforcement learning is to learn beneficial behaviors in a shared environment with other simultaneously learning agents.
Deep RL With Information Constrained Policies: Generalization in Continuous Control
We focus on the model-free reinforcement learning (RL) setting and formalize our approach in terms of an information-theoretic constraint on the complexity of learned policies.
In the Eye of the Beholder: Gaze and Actions in First Person Video
Moving beyond the dataset, we propose a novel deep model for joint gaze estimation and action recognition in FPV.
Coagent Networks Revisited
This work is aiming to discuss and close some of the gaps in the literature on models using options (and more generally coagents).
On the Role of Weight Sharing During Deep Option Learning
In this work we note that while this key assumption of the policy gradient theorems of option-critic holds in the tabular case, it is always violated in practice for the deep function approximation setting.
Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video
Motivated by this, we adopt intentional hand movement as a future representation and propose a novel deep network that jointly models and predicts the egocentric hand motion, interaction hotspots and future action.
CAPACITY-LIMITED REINFORCEMENT LEARNING: APPLICATIONS IN DEEP ACTOR-CRITIC METHODS FOR CONTINUOUS CONTROL
We formalize this type of bounded rationality in terms of an information-theoretic constraint on the complexity of policies that agents seek to learn.
Attention Distillation for Learning Video Representations
To this end, we make use of attention modules that learn to highlight regions in the video and aggregate features for recognition.
Ranked #36 on
Action Recognition
on UCF101
Learning Hierarchical Teaching Policies for Cooperative Agents
Collective learning can be greatly enhanced when agents effectively exchange knowledge with their peers.
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
In this work we propose a new conceptualization of the continual learning problem in terms of a temporally symmetric trade-off between transfer and interference that can be optimized by enforcing gradient alignment across examples.
Learning Abstract Options
Building systems that autonomously create temporal abstractions from data is a key challenge in scaling learning and planning in reinforcement learning.
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video
We address the task of jointly determining what a person is doing and where they are looking based on the analysis of video captured by a headworn camera.
Learning to Teach in Cooperative Multiagent Reinforcement Learning
The problem of teaching to improve agent learning has been investigated by prior works, but these approaches make assumptions that prevent application of teaching to general multiagent problems, or require domain expertise for problems they can apply to.
Faster Reinforcement Learning with Expert State Sequences
In this paper, we consider a realistic and more difficult sce- nario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are not available.
The Eigenoption-Critic Framework
Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration.
Efficient Exploration
Hierarchical Reinforcement Learning
+1
Eigenoption Discovery through the Deep Successor Representation
Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning.
Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions
We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.
Socially Aware Motion Planning with Deep Reinforcement Learning
For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e. g., passing on the right).
Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning
Finding feasible, collision-free paths for multiagent systems can be challenging, particularly in non-communicating scenarios where each agent's intent (e. g. goal) is unobservable to the others.
Multiagent Systems
Stick-Breaking Policy Learning in Dec-POMDPs
Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs).
Dynamic Clustering via Asymptotics of the Dependent Dirichlet Process Mixture
This paper presents a novel algorithm, based upon the dependent Dirichlet process mixture model (DDPMM), for clustering batch-sequential data containing an unknown number of evolving clusters.
