Search Results for author: Miao Liu

Found 52 papers, 16 papers with code

ADR-BC: Adversarial Density Weighted Regression Behavior Cloning

no code implementations28 May 2024 Ziqi Zhang, Zifeng Zhuang, Donglin Wang, Jingzehua Xu, Miao Liu, Shuai Zhang

Meanwhile, as a one-step behavior cloning framework, ADR-BC avoids the cumulative bias associated with multi-step RL frameworks.

Imitation Learning Q-Learning +2

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning

no code implementations24 May 2024 Shuai Zhang, Heshan Devaka Fernando, Miao Liu, Keerthiram Murugesan, Songtao Lu, Pin-Yu Chen, Tianyi Chen, Meng Wang

This paper studies the transfer reinforcement learning (RL) problem where multiple RL problems have different reward functions but share the same underlying transition dynamics.

Q-Learning Reinforcement Learning (RL) +1

BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

no code implementations CVPR 2024 Yunhao Ge, Yihe Tang, Jiashu Xu, Cem Gokmen, Chengshu Li, Wensi Ai, Benjamin Jose Martinez, Arman Aydin, Mona Anvari, Ayush K Chakravarthy, Hong-Xing Yu, Josiah Wong, Sanjana Srivastava, Sharon Lee, Shengxin Zha, Laurent Itti, Yunzhu Li, Roberto Martín-Martín, Miao Liu, Pengchuan Zhang, Ruohan Zhang, Li Fei-Fei, Jiajun Wu

We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models, based on the newly developed embodied AI benchmark, BEHAVIOR-1K.

Scene Understanding

A Neuro-Symbolic Approach to Multi-Agent RL for Interpretability and Probabilistic Decision Making

no code implementations21 Feb 2024 Chitra Subramanian, Miao Liu, Naweed Khan, Jonathan Lenchner, Aporva Amarnath, Sarathkrishna Swaminathan, Ryan Riegel, Alexander Gray

To enable decision-making under uncertainty and partial observability, we developed a novel probabilistic neuro-symbolic framework, Probabilistic Logical Neural Networks (PLNN), which combines the capabilities of logical reasoning with probabilistic graphical models.

Decision Making Decision Making Under Uncertainty +2

Context-Former: Stitching via Latent Conditioned Sequence Modeling

no code implementations29 Jan 2024 Ziqi Zhang, Jingzehua Xu, Jinxin Liu, Zifeng Zhuang, Donglin Wang, Miao Liu, Shuai Zhang

Offline reinforcement learning (RL) algorithms can learn better decision-making compared to behavior policies by stitching the suboptimal trajectories to derive more optimal ones.

D4RL Imitation Learning +2

The Audio-Visual Conversational Graph: From an Egocentric-Exocentric Perspective

no code implementations CVPR 2024 Wenqi Jia, Miao Liu, Hao Jiang, Ishwarya Ananthabhotla, James M. Rehg, Vamsi Krishna Ithapu, Ruohan Gao

We propose a unified multi-modal framework -- Audio-Visual Conversational Attention (AV-CONV), for the joint prediction of conversation behaviors -- speaking and listening -- for both the camera wearer as well as all other social partners present in the egocentric video.

LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning

no code implementations6 Dec 2023 Bolin Lai, Xiaoliang Dai, Lawrence Chen, Guan Pang, James M. Rehg, Miao Liu

Additionally, existing diffusion-based image manipulation models are sub-optimal in controlling the state transition of an action in egocentric image pixel space because of the domain gap.

Image Manipulation Language Modelling +1

Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives

2 code implementations CVPR 2024 Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, Eugene Byrne, Zach Chavis, Joya Chen, Feng Cheng, Fu-Jen Chu, Sean Crane, Avijit Dasgupta, Jing Dong, Maria Escobar, Cristhian Forigua, Abrham Gebreselasie, Sanjay Haresh, Jing Huang, Md Mohaiminul Islam, Suyog Jain, Rawal Khirodkar, Devansh Kukreja, Kevin J Liang, Jia-Wei Liu, Sagnik Majumder, Yongsen Mao, Miguel Martin, Effrosyni Mavroudi, Tushar Nagarajan, Francesco Ragusa, Santhosh Kumar Ramakrishnan, Luigi Seminara, Arjun Somayazulu, Yale Song, Shan Su, Zihui Xue, Edward Zhang, Jinxu Zhang, Angela Castillo, Changan Chen, Xinzhu Fu, Ryosuke Furuta, Cristina Gonzalez, Prince Gupta, Jiabo Hu, Yifei HUANG, Yiming Huang, Weslie Khoo, Anush Kumar, Robert Kuo, Sach Lakhavani, Miao Liu, Mi Luo, Zhengyi Luo, Brighid Meredith, Austin Miller, Oluwatumininu Oguntola, Xiaqing Pan, Penny Peng, Shraman Pramanick, Merey Ramazanova, Fiona Ryan, Wei Shan, Kiran Somasundaram, Chenan Song, Audrey Southerland, Masatoshi Tateno, Huiyu Wang, Yuchen Wang, Takuma Yagi, Mingfei Yan, Xitong Yang, Zecheng Yu, Shengxin Cindy Zha, Chen Zhao, Ziwei Zhao, Zhifan Zhu, Jeff Zhuo, Pablo Arbelaez, Gedas Bertasius, David Crandall, Dima Damen, Jakob Engel, Giovanni Maria Farinella, Antonino Furnari, Bernard Ghanem, Judy Hoffman, C. V. Jawahar, Richard Newcombe, Hyun Soo Park, James M. Rehg, Yoichi Sato, Manolis Savva, Jianbo Shi, Mike Zheng Shou, Michael Wray

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.

Video Understanding

On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$-Greedy Exploration

no code implementations24 Oct 2023 Shuai Zhang, Hongkang Li, Meng Wang, Miao Liu, Pin-Yu Chen, Songtao Lu, Sijia Liu, Keerthiram Murugesan, Subhajit Chaudhury

This paper provides the first theoretical convergence and sample complexity analysis of the practical setting of DQNs with $\epsilon$-greedy policy.

Q-Learning

Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi

1 code implementation20 Aug 2023 Hadi Nekoei, Xutong Zhao, Janarthanan Rajendran, Miao Liu, Sarath Chandar

In this work, we show empirically that state-of-the-art ZSC algorithms have poor performance when paired with agents trained with different learning methods, and they require millions of interaction samples to adapt to these new partners.

Game of Hanabi Multi-agent Reinforcement Learning +1

Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation

no code implementations6 May 2023 Bolin Lai, Fiona Ryan, Wenqi Jia, Miao Liu, James M. Rehg

Motivated by this observation, we introduce the first model that leverages both the video and audio modalities for egocentric gaze anticipation.

Representation Learning

Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks

no code implementations6 Feb 2023 Shuai Zhang, Meng Wang, Pin-Yu Chen, Sijia Liu, Songtao Lu, Miao Liu

Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and storage costs.

Sparse Learning

Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games

no code implementations16 Dec 2022 Bolin Lai, Hongxin Zhang, Miao Liu, Aryan Pariani, Fiona Ryan, Wenqi Jia, Shirley Anugrah Hayati, James M. Rehg, Diyi Yang

We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes.

Persuasion Strategies

Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria

no code implementations28 Oct 2022 Dong-Ki Kim, Matthew Riemer, Miao Liu, Jakob N. Foerster, Gerald Tesauro, Jonathan P. How

By directly comparing active equilibria to Nash equilibria in these examples, we find that active equilibria find more effective solutions than Nash equilibria, concluding that an active equilibrium is the desired solution for multiagent learning settings.

Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach

1 code implementation23 Oct 2022 Heshan Fernando, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, Tianyi Chen

Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and accuracy; or, in multi-task learning where multiple tasks are optimized jointly, sharing inductive bias between them.

Fairness Inductive Bias +1

In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation

no code implementations8 Aug 2022 Bolin Lai, Miao Liu, Fiona Ryan, James M. Rehg

To this end, we design the transformer encoder to embed the global context as one additional visual token and further propose a novel Global-Local Correlation (GLC) module to explicitly model the correlation of the global token and each local token.

Gaze Estimation

Influencing Long-Term Behavior in Multiagent Reinforcement Learning

1 code implementation7 Mar 2022 Dong-Ki Kim, Matthew Riemer, Miao Liu, Jakob N. Foerster, Michael Everett, Chuangchuang Sun, Gerald Tesauro, Jonathan P. How

An effective approach that has recently emerged for addressing this non-stationarity is for each agent to anticipate the learning of other agents and influence the evolution of future policies towards desirable behavior for its own benefit.

reinforcement-learning Reinforcement Learning (RL)

Hierarchical Reinforcement Learning with AI Planning Models

1 code implementation1 Mar 2022 JunKyu Lee, Michael Katz, Don Joven Agravante, Miao Liu, Geraud Nangue Tasse, Tim Klinger, Shirin Sohrabi

Our approach defines options in hierarchical reinforcement learning (HRL) from AIP operators by establishing a correspondence between the state transition model of AI planning problem and the abstract state transition system of a Markov Decision Process (MDP).

Decision Making Hierarchical Reinforcement Learning +2

Ego4D: Around the World in 3,000 Hours of Egocentric Video

8 code implementations CVPR 2022 Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei HUANG, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Ziwei Zhao, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Christian Fuegen, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, Jitendra Malik

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.

De-identification Ethics

Learning Multi-Objective Curricula for Robotic Policy Learning

1 code implementation6 Oct 2021 Jikun Kang, Miao Liu, Abhinav Gupta, Chris Pal, Xue Liu, Jie Fu

Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL).

Reinforcement Learning (RL)

Interpreting Reinforcement Policies through Local Behaviors

no code implementations29 Sep 2021 Ronny Luss, Amit Dhurandhar, Miao Liu

Many works in explainable AI have focused on explaining black-box classification models.

Reinforcement Learning (RL)

Context-Specific Representation Abstraction for Deep Option Learning

1 code implementation20 Sep 2021 Marwa Abdulhai, Dong-Ki Kim, Matthew Riemer, Miao Liu, Gerald Tesauro, Jonathan P. How

Hierarchical reinforcement learning has focused on discovering temporally extended actions, such as options, that can provide benefits in problems requiring extensive exploration.

Hierarchical Reinforcement Learning

Egocentric Activity Recognition and Localization on a 3D Map

no code implementations20 May 2021 Miao Liu, Lingni Ma, Kiran Somasundaram, Yin Li, Kristen Grauman, James M. Rehg, Chao Li

Given a video captured from a first person perspective and the environment context of where the video is recorded, can we recognize what the person is doing and identify where the action occurs in the 3D space?

Action Localization Action Recognition +2

4D Human Body Capture from Egocentric Video via 3D Scene Grounding

no code implementations26 Nov 2020 Miao Liu, Dexin Yang, Yan Zhang, Zhaopeng Cui, James M. Rehg, Siyu Tang

We introduce a novel task of reconstructing a time series of second-person 3D human body meshes from monocular egocentric videos.

Time Series Time Series Analysis

Consolidation via Policy Information Regularization in Deep RL for Multi-Agent Games

no code implementations23 Nov 2020 Tyler Malloy, Tim Klinger, Miao Liu, Matthew Riemer, Gerald Tesauro, Chris R. Sims

This paper introduces an information-theoretic constraint on learned policy complexity in the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) reinforcement learning algorithm.

Continual Learning Continuous Control +2

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

1 code implementation31 Oct 2020 Dong-Ki Kim, Miao Liu, Matthew Riemer, Chuangchuang Sun, Marwa Abdulhai, Golnaz Habibi, Sebastian Lopez-Cot, Gerald Tesauro, Jonathan P. How

A fundamental challenge in multiagent reinforcement learning is to learn beneficial behaviors in a shared environment with other simultaneously learning agents.

reinforcement-learning Reinforcement Learning (RL)

Deep RL With Information Constrained Policies: Generalization in Continuous Control

no code implementations9 Oct 2020 Tyler Malloy, Chris R. Sims, Tim Klinger, Miao Liu, Matthew Riemer, Gerald Tesauro

We focus on the model-free reinforcement learning (RL) setting and formalize our approach in terms of an information-theoretic constraint on the complexity of learned policies.

Continuous Control reinforcement-learning +1

In the Eye of the Beholder: Gaze and Actions in First Person Video

no code implementations31 May 2020 Yin Li, Miao Liu, James M. Rehg

Moving beyond the dataset, we propose a novel deep model for joint gaze estimation and action recognition in FPV.

Action Recognition Gaze Estimation

Coagent Networks Revisited

1 code implementation28 Jan 2020 Modjtaba Shokrian Zini, Mohammad Pedramfar, Matthew Riemer, Ahmadreza Moradipari, Miao Liu

Coagent networks formalize the concept of arbitrary networks of stochastic agents that collaborate to take actions in a reinforcement learning environment.

Hierarchical Reinforcement Learning reinforcement-learning

On the Role of Weight Sharing During Deep Option Learning

no code implementations31 Dec 2019 Matthew Riemer, Ignacio Cases, Clemens Rosenbaum, Miao Liu, Gerald Tesauro

In this work we note that while this key assumption of the policy gradient theorems of option-critic holds in the tabular case, it is always violated in practice for the deep function approximation setting.

Atari Games

Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video

1 code implementation ECCV 2020 Miao Liu, Siyu Tang, Yin Li, James Rehg

Motivated by this, we adopt intentional hand movement as a future representation and propose a novel deep network that jointly models and predicts the egocentric hand motion, interaction hotspots and future action.

Action Anticipation Human-Object Interaction Detection

CAPACITY-LIMITED REINFORCEMENT LEARNING: APPLICATIONS IN DEEP ACTOR-CRITIC METHODS FOR CONTINUOUS CONTROL

no code implementations25 Sep 2019 Tyler James Malloy, Matthew Riemer, Miao Liu, Tim Klinger, Gerald Tesauro, Chris R. Sims

We formalize this type of bounded rationality in terms of an information-theoretic constraint on the complexity of policies that agents seek to learn.

Continuous Control reinforcement-learning +1

Attention Distillation for Learning Video Representations

no code implementations5 Apr 2019 Miao Liu, Xin Chen, Yun Zhang, Yin Li, James M. Rehg

To this end, we make use of attention modules that learn to highlight regions in the video and aggregate features for recognition.

Action Recognition Video Recognition

Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference

3 code implementations ICLR 2019 Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, Gerald Tesauro

In this work we propose a new conceptualization of the continual learning problem in terms of a temporally symmetric trade-off between transfer and interference that can be optimized by enforcing gradient alignment across examples.

Continual Learning Meta-Learning

Learning Abstract Options

no code implementations NeurIPS 2018 Matthew Riemer, Miao Liu, Gerald Tesauro

Building systems that autonomously create temporal abstractions from data is a key challenge in scaling learning and planning in reinforcement learning.

In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video

no code implementations ECCV 2018 Yin Li, Miao Liu, James M. Rehg

We address the task of jointly determining what a person is doing and where they are looking based on the analysis of video captured by a headworn camera.

Action Recognition Gaze Estimation +1

Learning to Teach in Cooperative Multiagent Reinforcement Learning

no code implementations20 May 2018 Shayegan Omidshafiei, Dong-Ki Kim, Miao Liu, Gerald Tesauro, Matthew Riemer, Christopher Amato, Murray Campbell, Jonathan P. How

The problem of teaching to improve agent learning has been investigated by prior works, but these approaches make assumptions that prevent application of teaching to general multiagent problems, or require domain expertise for problems they can apply to.

reinforcement-learning Reinforcement Learning (RL)

Faster Reinforcement Learning with Expert State Sequences

no code implementations ICLR 2018 Xiaoxiao Guo, Shiyu Chang, Mo Yu, Miao Liu, Gerald Tesauro

In this paper, we consider a realistic and more difficult sce- nario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are not available.

Imitation Learning reinforcement-learning +1

The Eigenoption-Critic Framework

no code implementations11 Dec 2017 Miao Liu, Marlos C. Machado, Gerald Tesauro, Murray Campbell

Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration.

Efficient Exploration Hierarchical Reinforcement Learning +1

Eigenoption Discovery through the Deep Successor Representation

1 code implementation ICLR 2018 Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, Murray Campbell

Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning.

Atari Games reinforcement-learning +2

Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

no code implementations24 Jul 2017 Miao Liu, Kavinayan Sivakumar, Shayegan Omidshafiei, Christopher Amato, Jonathan P. How

We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.

Decision Making Decision Making Under Uncertainty

Socially Aware Motion Planning with Deep Reinforcement Learning

2 code implementations26 Mar 2017 Yu Fan Chen, Michael Everett, Miao Liu, Jonathan P. How

For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e. g., passing on the right).

Autonomous Navigation Motion Planning +3

Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning

no code implementations26 Sep 2016 Yu Fan Chen, Miao Liu, Michael Everett, Jonathan P. How

Finding feasible, collision-free paths for multiagent systems can be challenging, particularly in non-communicating scenarios where each agent's intent (e. g. goal) is unobservable to the others.

Multiagent Systems

Stick-Breaking Policy Learning in Dec-POMDPs

no code implementations1 May 2015 Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, Jonathan P. How

Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs).

Dynamic Clustering via Asymptotics of the Dependent Dirichlet Process Mixture

1 code implementation NeurIPS 2013 Trevor Campbell, Miao Liu, Brian Kulis, Jonathan P. How, Lawrence Carin

This paper presents a novel algorithm, based upon the dependent Dirichlet process mixture model (DDPMM), for clustering batch-sequential data containing an unknown number of evolving clusters.

Clustering

Cannot find the paper you are looking for? You can Submit a new open access paper.