no code implementations • 17 Dec 2024 • Chen Bao, Jiarui Xu, Xiaolong Wang, Abhinav Gupta, Homanga Bharadhwaj
How can we predict future interaction trajectories of human hands in a scene given high-level colloquial task specifications in the form of natural language?
no code implementations • 24 Sep 2024 • Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, Sean Kirmani
To train the policy, we use an order of magnitude less robot interaction data compared to what the video prediction model was trained on.
no code implementations • 2 Sep 2024 • Zoey Chen, Zhao Mandi, Homanga Bharadhwaj, Mohit Sharma, Shuran Song, Abhishek Gupta, Vikash Kumar
By demonstrating the effectiveness of image-text generative models in diverse real-world robotic applications, our generative augmentation framework provides a scalable and efficient path for boosting generalization in robot learning at no extra human cost.
no code implementations • 2 May 2024 • Homanga Bharadhwaj, Roozbeh Mottaghi, Abhinav Gupta, Shubham Tulsiani
We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation: interacting with unseen objects in novel scenes without test-time adaptation.
no code implementations • 1 Dec 2023 • Homanga Bharadhwaj, Abhinav Gupta, Vikash Kumar, Shubham Tulsiani
We pursue the goal of developing robots that can interact zero-shot with generic unseen objects via a diverse repertoire of manipulation skills and show how passive human videos can serve as a rich source of data for learning such generalist robots.
no code implementations • 5 Sep 2023 • Homanga Bharadhwaj, Jay Vakil, Mohit Sharma, Abhinav Gupta, Shubham Tulsiani, Vikash Kumar
The grand aim of having a single robot that can manipulate arbitrary objects in diverse settings is at odds with the paucity of robotics datasets.
no code implementations • 28 May 2023 • Homanga Bharadhwaj, Abhinav Gupta, Shubham Tulsiani
Motivated by the intuitive understanding humans have about the space of possible interactions, and the ease with which they can generalize this understanding to previously unseen scenes, we develop an approach for learning visual affordances for guiding robot exploration.
no code implementations • 3 Feb 2023 • Homanga Bharadhwaj, Abhinav Gupta, Shubham Tulsiani, Vikash Kumar
Can we learn robot manipulation for everyday tasks, only by watching videos of humans doing arbitrary tasks in different unstructured settings?
no code implementations • 29 Dec 2022 • Riashat Islam, Samarth Sinha, Homanga Bharadhwaj, Samin Yeasar Arnob, Zhuoran Yang, Animesh Garg, Zhaoran Wang, Lihong Li, Doina Precup
Learning policies from fixed offline datasets is a key challenge to scale up reinforcement learning (RL) algorithms towards practical applications.
1 code implementation • 12 Dec 2022 • Zhao Mandi, Homanga Bharadhwaj, Vincent Moens, Shuran Song, Aravind Rajeswaran, Vikash Kumar
On a real robot setup, CACTI enables efficient training of a single policy that can perform 10 manipulation tasks involving kitchen objects, and is robust to varying layouts of distractors.
no code implementations • 18 Sep 2022 • Raj Ghugare, Homanga Bharadhwaj, Benjamin Eysenbach, Sergey Levine, Ruslan Salakhutdinov
In this work, we propose a single objective which jointly optimizes a latent-space model and policy to achieve high returns while remaining self-consistent.
no code implementations • ICLR 2022 • Homanga Bharadhwaj, Mohammad Babaeizadeh, Dumitru Erhan, Sergey Levine
We propose a modified objective for model-based RL that, in combination with mutual information maximization, allows us to learn representations and dynamics for visual model-based RL without reconstruction in a way that explicitly prioritizes functionally relevant factors.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
no code implementations • 12 Oct 2021 • Homanga Bharadhwaj
They are also likely going to fail and be incompliant with human preferences in increasingly subtle ways.
no code implementations • 25 Sep 2021 • Homanga Bharadhwaj, De-An Huang, Chaowei Xiao, Anima Anandkumar, Animesh Garg
We enable such unit tests through variations in a semantically-interpretable latent space of a generative model.
no code implementations • 18 Jan 2021 • Haoyu Xiong, Quanzhou Li, Yun-Chun Chen, Homanga Bharadhwaj, Samarth Sinha, Animesh Garg
Learning from visual data opens the potential to accrue a large range of manipulation behaviors by leveraging human demonstrations without specifying each of them mathematically, but rather through natural task specification.
no code implementations • 1 Jan 2021 • Riashat Islam, Samarth Sinha, Homanga Bharadhwaj, Samin Yeasar Arnob, Zhuoran Yang, Zhaoran Wang, Animesh Garg, Lihong Li, Doina Precup
Learning policies from fixed offline datasets is a key challenge to scale up reinforcement learning (RL) algorithms towards practical applications.
no code implementations • ICLR 2021 • Kevin Xie, Homanga Bharadhwaj, Danijar Hafner, Animesh Garg, Florian Shkurti
To quickly solve new tasks in complex environments, intelligent agents need to build up reusable knowledge.
no code implementations • ICLR 2021 • Homanga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine, Florian Shkurti, Animesh Garg
Safe exploration presents a major challenge in reinforcement learning (RL): when active data collection requires deploying partially trained policies, we must ensure that these policies avoid catastrophically unsafe regions, while still enabling trial and error learning.
4 code implementations • 19 Oct 2020 • Samarth Sinha, Homanga Bharadhwaj, Aravind Srinivas, Animesh Garg
While improvements in deep learning architectures have played a crucial role in improving the state of supervised and unsupervised learning in computer vision and natural language processing, neural network architecture choices for reinforcement learning remain relatively under-explored.
1 code implementation • 25 Sep 2020 • Yizhou Huang, Kevin Xie, Homanga Bharadhwaj, Florian Shkurti
Effective planning in model-based reinforcement learning (MBRL) and model-predictive control (MPC) relies on the accuracy of the learned dynamics model.
no code implementations • 2 Sep 2020 • Vikas Singh, Homanga Bharadhwaj, Nishchal K. Verma
Clustering techniques have been proved highly suc-cessful for Takagi-Sugeno (T-S) fuzzy model identification.
no code implementations • 1 Jul 2020 • Homanga Bharadhwaj, Dylan Turpin, Animesh Garg, Ashton Anderson
Under two conditions: papers that are released on arXiv before the review phase and papers that are not, we examine the correlation between the reputation of their authors with the review scores and acceptance decisions.
no code implementations • 15 Jun 2020 • Yatin Dandi, Homanga Bharadhwaj, Abhishek Kumar, Piyush Rai
Recent approaches, such as ALI and BiGAN frameworks, develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs.
no code implementations • 21 May 2020 • Homanga Bharadhwaj, Animesh Garg, Florian Shkurti
We target the challenging problem of policy learning from initial and goal states specified as images, and do not assume any access to the underlying ground-truth states of the robot and the environment.
2 code implementations • ECCV 2020 • Timo Milbich, Karsten Roth, Homanga Bharadhwaj, Samarth Sinha, Yoshua Bengio, Björn Ommer, Joseph Paul Cohen
Visual Similarity plays an important role in many computer vision applications.
Ranked #13 on
Metric Learning
on CUB-200-2011
(using extra training data)
1 code implementation • L4DC 2020 • Homanga Bharadhwaj, Kevin Xie, Florian Shkurti
Recent works in high-dimensional model-predictive control and model-based reinforcement learning with learned dynamics and reward models have resorted to population-based optimization methods, such as the Cross-Entropy Method (CEM), for planning a sequence of actions.
Model-based Reinforcement Learning
Model Predictive Control
+1
1 code implementation • 10 Mar 2020 • Samarth Sinha, Homanga Bharadhwaj, Anirudh Goyal, Hugo Larochelle, Animesh Garg, Florian Shkurti
Although deep learning models have achieved state-of-the-art performance on a number of vision tasks, generalization over high dimensional multi-modal data, and reliable predictive uncertainty estimation are still active areas of research.
no code implementations • 19 Nov 2019 • Homanga Bharadhwaj, Shoichiro Yamaguchi, Shin-ichi Maeda
Efficiently transferring learned policies to an unknown environment with changes in dynamics configurations in the presence of motor noise is very important for operating robots in the real world, and our work is a novel attempt in that direction.
1 code implementation • 7 Jun 2019 • Varun Khare, Divyat Mahajan, Homanga Bharadhwaj, Vinay Verma, Piyush Rai
Our approach is based on end-to-end learning of the class distributions of seen classes and unseen classes.
Ranked #1 on
Zero-Shot Learning
on CUB-200 - 0-Shot Learning
(using extra training data)
no code implementations • 11 Oct 2018 • Homanga Bharadhwaj, Zihan Wang, Yoshua Bengio, Liam Paull
Learning effective visuomotor policies for robots purely from data is challenging, but also appealing since a learning-based system should not require manual tuning or calibration.
no code implementations • 17 Jul 2018 • Homanga Bharadhwaj
Relationships between the images are identified by the model and layer-wise relevance propagation is used to infer pixel-level details of the images that may have significantly informed the model's choice.
no code implementations • 17 Jul 2018 • Homanga Bharadhwaj, Shruti Joshi
Recommendation systems are an integral part of Artificial Intelligence (AI) and have become increasingly important in the growing age of commercialization in AI.