1 code implementation • 3 Oct 2023 • Anish Bhattacharya, Ratnesh Madaan, Fernando Cladera, Sai Vemprala, Rogerio Bonatti, Kostas Daniilidis, Ashish Kapoor, Vijay Kumar, Nikolai Matni, Jayesh K. Gupta
We present EvDNeRF, a pipeline for generating event data and training an event-based dynamic NeRF, for the purpose of faithfully reconstructing eventstreams on scenes with rigid and non-rigid deformations that may be too fast to capture with a standard camera.
1 code implementation • 2 Oct 2023 • Sai Vemprala, Shuhang Chen, Abhinav Shukla, Dinesh Narayanan, Ashish Kapoor
In addition, the modular design enables various deep ML components and existing foundation models to be easily usable in a wider variety of robot-centric problems.
1 code implementation • ICCV 2023 • Yao Wei, Yanchao Sun, Ruijie Zheng, Sai Vemprala, Rogerio Bonatti, Shuhang Chen, Ratnesh Madaan, Zhongjie Ba, Ashish Kapoor, Shuang Ma
We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning.
no code implementations • 7 Mar 2023 • Yue Meng, Sai Vemprala, Rogerio Bonatti, Chuchu Fan, Ashish Kapoor
In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion.
1 code implementation • 20 Feb 2023 • Sai Vemprala, Rogerio Bonatti, Arthur Bucker, Ashish Kapoor
This paper presents an experimental study regarding the use of OpenAI's ChatGPT for robotics applications.
1 code implementation • 24 Jan 2023 • Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, Aditya Grover
We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings.
no code implementations • 24 Jan 2023 • Yanchao Sun, Shuang Ma, Ratnesh Madaan, Rogerio Bonatti, Furong Huang, Ashish Kapoor
Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining representations without explicit labels.
1 code implementation • 18 Nov 2022 • Jiachen Lei, Shuang Ma, Zhongjie Ba, Sai Vemprala, Ashish Kapoor, Kui Ren
In this report, we present our approach and empirical results of applying masked autoencoders in two egocentric video understanding tasks, namely, Object State Change Classification and PNR Temporal Localization, of Ego4D Challenge 2022.
1 code implementation • 28 Oct 2022 • Jayesh K. Gupta, Sai Vemprala, Ashish Kapoor
We evaluate our framework on a variety of systems and show that message passing allows coordination between multiple modules over time for accurate predictions and in certain cases, enables zero-shot generalization to new system configurations.
no code implementations • 22 Sep 2022 • Benoit Guillard, Sai Vemprala, Jayesh K. Gupta, Ondrej Miksik, Vibhav Vineet, Pascal Fua, Ashish Kapoor
Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling.
no code implementations • 22 Sep 2022 • Rogerio Bonatti, Sai Vemprala, Shuang Ma, Felipe Frujeri, Shuhang Chen, Ashish Kapoor
Robotics has long been a field riddled with complex systems architectures whose modules and connections, whether traditional or learning-based, require significant human expertise and prior knowledge.
2 code implementations • 4 Aug 2022 • Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Sai Vemprala, Rogerio Bonatti
Natural language is one of the most intuitive ways to express human intent.
no code implementations • 29 Jul 2022 • Wenhao Luo, Wen Sun, Ashish Kapoor
In particular, the framework 1) extends control barrier functions (CBFs) in a stochastic setting to achieve provable high-probability safety under uncertainty during model learning and 2) integrates an optimism-based exploration strategy to efficiently guide the safe exploration process with learned dynamics for \emph{near optimal} control performance.
no code implementations • 25 Mar 2022 • Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Rogerio Bonatti
However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands.
no code implementations • 25 Jun 2021 • Daniel McDuff, Yale Song, Jiyoung Lee, Vibhav Vineet, Sai Vemprala, Nicholas Gyde, Hadi Salman, Shuang Ma, Kwanghoon Sohn, Ashish Kapoor
The ability to perform causal and counterfactual reasoning are central properties of human intelligence.
1 code implementation • 7 Jun 2021 • Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, Ashish Kapoor, Aleksander Madry
We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation.
1 code implementation • NeurIPS 2021 • Sai Vemprala, Sami Mian, Ashish Kapoor
Event-based cameras are dynamic vision sensors that provide asynchronous measurements of changes in per-pixel brightness at a microsecond level.
2 code implementations • NeurIPS 2021 • Hadi Salman, Andrew Ilyas, Logan Engstrom, Sai Vemprala, Aleksander Madry, Ashish Kapoor
We study a class of realistic computer vision settings wherein one can influence the design of the objects being recognized.
no code implementations • 17 Dec 2020 • Ashish Kapoor
Commercial aviation is one of the biggest contributors towards climate change.
2 code implementations • NeurIPS 2020 • Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, Aleksander Madry
Typically, better pre-trained models yield better transfer results, suggesting that initial accuracy is a key aspect of transfer learning performance.
1 code implementation • 31 Mar 2020 • Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, Sebastian Scherer
We present a challenging dataset, the TartanAir, for robot navigation task and more.
Robotics
2 code implementations • 12 Mar 2020 • Ratnesh Madaan, Nicholas Gyde, Sai Vemprala, Matthew Brown, Keiko Nagami, Tim Taubner, Eric Cristofalo, Davide Scaramuzza, Mac Schwager, Ashish Kapoor
Autonomous drone racing is a challenging research problem at the intersection of computer vision, planning, state estimation, and control.
4 code implementations • NeurIPS 2020 • Hadi Salman, Ming-Jie Sun, Greg Yang, Ashish Kapoor, J. Zico Kolter
We present a method for provably defending any pretrained image classifier against $\ell_p$ adversarial attacks.
no code implementations • 4 Dec 2019 • Kiwan Maeng, Iskender Kushan, Brandon Lucia, Ashish Kapoor
We propose a framework to collect stratospheric data by releasing a contrail of tiny sensor devices as a weather balloon ascends.
2 code implementations • 1 Dec 2019 • Dean Zadok, Daniel McDuff, Ashish Kapoor
Positive affect has been linked to increased interest, curiosity and satisfaction in human learning.
2 code implementations • 16 Sep 2019 • Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors.
2 code implementations • NeurIPS 2019 • Aditya Grover, Jiaming Song, Alekh Agarwal, Kenneth Tran, Ashish Kapoor, Eric Horvitz, Stefano Ermon
A standard technique to correct this bias is importance sampling, where samples from the model are weighted by the likelihood ratio under model and true distributions.
1 code implementation • NeurIPS 2019 • Daniel McDuff, Shuang Ma, Yale Song, Ashish Kapoor
Models that are learned from real-world data are often biased because the data used to train them is biased.
4 code implementations • 15 May 2019 • Dean Zadok, Tom Hirshberg, Amir Biran, Kira Radinsky, Ashish Kapoor
This paper describes the exploration and learnings during the process of developing a self-driving algorithm in simulation, followed by deployment on a real car.
Robotics
no code implementations • ICLR 2019 • Daniel McDuff, Ashish Kapoor
The human autonomic nervous system has evolved over millions of years and is essential for survival and responding to threats.
no code implementations • 11 Apr 2019 • Sara Beery, Yang Liu, Dan Morris, Jim Piavis, Ashish Kapoor, Markus Meister, Neel Joshi, Pietro Perona
The ability to detect and classify rare occurrences in images has important applications - for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars.
no code implementations • ICLR Workshop DeepGenStruct 2019 • Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric Horvitz, Stefano Ermon
A standard technique to correct this bias is by importance weighting samples from the model by the likelihood ratio under the model and true distributions.
no code implementations • ICLR 2019 • Daniel McDuff, Roger Cheng, Ashish Kapoor
Machine learned models exhibit bias, often because the datasets used to train them are biased.
no code implementations • 27 Sep 2018 • Felix Berkenkamp, Debadeepta Dey, Ashish Kapoor
Deep reinforcement learning has enabled robots to complete complex tasks in simulation.
no code implementations • ECCV 2018 • Benjamin Hepp, Debadeepta Dey, Sudipta N. Sinha, Ashish Kapoor, Neel Joshi, Otmar Hilliges
We propose to learn a better utility function that predicts the usefulness of future viewpoints.
2 code implementations • 25 May 2018 • Daniel McDuff, Ashish Kapoor
As people learn to navigate the world, autonomic nervous system (e. g., "fight or flight") responses provide intrinsic feedback about the potential consequence of action choices (e. g., becoming nervous when close to a cliff edge or driving fast around a bend.)
1 code implementation • 23 Feb 2018 • Shromona Ghosh, Felix Berkenkamp, Gireeja Ranade, Shaz Qadeer, Ashish Kapoor
We specify safety constraints using logic and exploit structure in the problem in order to test the system for adversarial counter examples that violate the safety specifications.
1 code implementation • 26 Sep 2017 • Emmanouil Antonios Platanios, Ashish Kapoor, Eric Horvitz
Structured prediction is ubiquitous in applications of machine learning such as knowledge extraction and natural language processing.
no code implementations • ICML 2017 • Wen Sun, Debadeepta Dey, Ashish Kapoor
To address this problem, we first study online convex programming in the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.
25 code implementations • 15 May 2017 • Shital Shah, Debadeepta Dey, Chris Lovett, Ashish Kapoor
Developing and testing algorithms for autonomous vehicles in real world is an expensive and time consuming process.
no code implementations • ICCV 2017 • Mike Roberts, Debadeepta Dey, Anh Truong, Sudipta Sinha, Shital Shah, Ashish Kapoor, Pat Hanrahan, Neel Joshi
Drones equipped with cameras are emerging as a powerful tool for large-scale aerial 3D scanning, but existing automatic flight planners do not exploit all available information about the scene, and can therefore produce inaccurate and incomplete 3D models.
no code implementations • 13 Nov 2016 • Sanjiban Choudhury, Ashish Kapoor, Gireeja Ranade, Debadeepta Dey
The budgeted information gathering problem - where a robot with a fixed fuel budget is required to maximize the amount of information gathered from the world - appears in practice across a wide range of applications in autonomous exploration and inspection with mobile robots.
no code implementations • 17 Oct 2016 • Wen Sun, Debadeepta Dey, Ashish Kapoor
To address this problem, we first study the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.
no code implementations • 16 Sep 2016 • Wen Sun, Niteesh Sood, Debadeepta Dey, Gireeja Ranade, Siddharth Prakash, Ashish Kapoor
This paper explores the problem of path planning under uncertainty.
no code implementations • NeurIPS 2016 • Nathan Wiebe, Ashish Kapoor, Krysta M. Svore
We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model.
no code implementations • 20 Nov 2015 • Nathan Wiebe, Christopher Granade, Ashish Kapoor, Krysta M. Svore
We provide a method for approximating Bayesian inference using rejection sampling.
no code implementations • 25 Oct 2015 • Dorsa Sadigh, Ashish Kapoor
In this paper, we propose a new logic, Probabilistic Signal Temporal Logic (PrSTL), as an expressive language to define the stochastic properties, and enforce probabilistic guarantees on them.
no code implementations • 9 Jul 2015 • Nathan Wiebe, Ashish Kapoor, Christopher Granade, Krysta M. Svore
We present an efficient classical algorithm for training deep Boltzmann machines (DBMs) that uses rejection sampling in concert with variational approximations to estimate the gradients of the training objective function.
no code implementations • 23 Jan 2015 • Ashish Kapoor, E. Paxon Frady, Stefanie Jegelka, William B. Kristan, Eric Horvitz
We introduce and study methods for inferring and learning from correspondences among neurons.
no code implementations • 10 Dec 2014 • Nathan Wiebe, Ashish Kapoor, Krysta M. Svore
In recent years, deep learning has had a profound impact on machine learning and artificial intelligence.
no code implementations • CVPR 2014 • Huixuan Tang, Neel Joshi, Ashish Kapoor
The biggest hurdles to these efforts are: 1) the difficulty of generalizing across diverse types of distortions and 2) collecting the enormity of human scored training data that is needed to learn the measure.
no code implementations • 23 Jan 2014 • Jonathan Huang, Ashish Kapoor, Carlos Guestrin
Simultaneously addressing all of these challenges i. e., designing a compactly representable model which is amenable to efficient inference and can be learned using partial ranking data is a difficult task, but is necessary if we would like to scale to problems with nontrivial size.
2 code implementations • 9 Jan 2014 • Nathan Wiebe, Ashish Kapoor, Krysta Svore
In the worst case, our quantum algorithms lead to polynomial reductions in query complexity relative to the corresponding classical algorithm.
Quantum Physics
no code implementations • NeurIPS 2012 • Ashish Kapoor, Raajay Viswanathan, Prateek Jain
The two key benefits of the model are that a) it can naturally handle datasets that have missing labels and b) it can also measure uncertainty in prediction.
no code implementations • NeurIPS 2009 • Ashish Kapoor, Eric Horvitz
There has been a clear distinction between induction or training time and diagnosis time active information acquisition.