Search Results for author: Ashish Kapoor

Found 55 papers, 25 papers with code

EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields

1 code implementation3 Oct 2023 Anish Bhattacharya, Ratnesh Madaan, Fernando Cladera, Sai Vemprala, Rogerio Bonatti, Kostas Daniilidis, Ashish Kapoor, Vijay Kumar, Nikolai Matni, Jayesh K. Gupta

We present EvDNeRF, a pipeline for generating event data and training an event-based dynamic NeRF, for the purpose of faithfully reconstructing eventstreams on scenes with rigid and non-rigid deformations that may be too fast to capture with a standard camera.

GRID: A Platform for General Robot Intelligence Development

1 code implementation2 Oct 2023 Sai Vemprala, Shuhang Chen, Abhinav Shukla, Dinesh Narayanan, Ashish Kapoor

In addition, the modular design enables various deep ML components and existing foundation models to be easily usable in a wider variety of robot-centric problems.

Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training

1 code implementation ICCV 2023 Yao Wei, Yanchao Sun, Ruijie Zheng, Sai Vemprala, Rogerio Bonatti, Shuhang Chen, Ratnesh Madaan, Zhongjie Ba, Ashish Kapoor, Shuang Ma

We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning.

Decision Making

ConBaT: Control Barrier Transformer for Safe Policy Learning

no code implementations7 Mar 2023 Yue Meng, Sai Vemprala, Rogerio Bonatti, Chuchu Fan, Ashish Kapoor

In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion.

Imitation Learning Model Predictive Control

ChatGPT for Robotics: Design Principles and Model Abilities

1 code implementation20 Feb 2023 Sai Vemprala, Rogerio Bonatti, Arthur Bucker, Ashish Kapoor

This paper presents an experimental study regarding the use of OpenAI's ChatGPT for robotics applications.

Mathematical Reasoning Prompt Engineering

SMART: Self-supervised Multi-task pretrAining with contRol Transformers

no code implementations24 Jan 2023 Yanchao Sun, Shuang Ma, Ratnesh Madaan, Rogerio Bonatti, Furong Huang, Ashish Kapoor

Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining representations without explicit labels.

Imitation Learning Reinforcement Learning (RL)

ClimaX: A foundation model for weather and climate

1 code implementation24 Jan 2023 Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, Aditya Grover

We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings.

Self-Supervised Learning Weather Forecasting

Masked Autoencoders for Egocentric Video Understanding @ Ego4D Challenge 2022

1 code implementation18 Nov 2022 Jiachen Lei, Shuang Ma, Zhongjie Ba, Sai Vemprala, Ashish Kapoor, Kui Ren

In this report, we present our approach and empirical results of applying masked autoencoders in two egocentric video understanding tasks, namely, Object State Change Classification and PNR Temporal Localization, of Ego4D Challenge 2022.

Object State Change Classification Temporal Localization +1

Learning Modular Simulations for Homogeneous Systems

1 code implementation28 Oct 2022 Jayesh K. Gupta, Sai Vemprala, Ashish Kapoor

We evaluate our framework on a variety of systems and show that message passing allows coordination between multiple modules over time for accurate predictions and in certain cases, enables zero-shot generalization to new system configurations.

Zero-shot Generalization

PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training

no code implementations22 Sep 2022 Rogerio Bonatti, Sai Vemprala, Shuang Ma, Felipe Frujeri, Shuhang Chen, Ashish Kapoor

Robotics has long been a field riddled with complex systems architectures whose modules and connections, whether traditional or learning-based, require significant human expertise and prior knowledge.

Learning to Simulate Realistic LiDARs

no code implementations22 Sep 2022 Benoit Guillard, Sai Vemprala, Jayesh K. Gupta, Ondrej Miksik, Vibhav Vineet, Pascal Fua, Ashish Kapoor

Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling.

LATTE: LAnguage Trajectory TransformEr

2 code implementations4 Aug 2022 Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Sai Vemprala, Rogerio Bonatti

Natural language is one of the most intuitive ways to express human intent.

Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions

no code implementations29 Jul 2022 Wenhao Luo, Wen Sun, Ashish Kapoor

In particular, the framework 1) extends control barrier functions (CBFs) in a stochastic setting to achieve provable high-probability safety under uncertainty during model learning and 2) integrates an optimism-based exploration strategy to efficiently guide the safe exploration process with learned dynamics for \emph{near optimal} control performance.

Decision Making Reinforcement Learning (RL) +1

Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers

no code implementations25 Mar 2022 Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Rogerio Bonatti

However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands.

Imitation Learning Text Generation

3DB: A Framework for Debugging Computer Vision Models

1 code implementation7 Jun 2021 Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, Ashish Kapoor, Aleksander Madry

We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation.

Representation Learning for Event-based Visuomotor Policies

1 code implementation NeurIPS 2021 Sai Vemprala, Sami Mian, Ashish Kapoor

Event-based cameras are dynamic vision sensors that provide asynchronous measurements of changes in per-pixel brightness at a microsecond level.

Representation Learning

Unadversarial Examples: Designing Objects for Robust Vision

2 code implementations NeurIPS 2021 Hadi Salman, Andrew Ilyas, Logan Engstrom, Sai Vemprala, Aleksander Madry, Ashish Kapoor

We study a class of realistic computer vision settings wherein one can influence the design of the objects being recognized.

BIG-bench Machine Learning

Helping Reduce Environmental Impact of Aviation with Machine Learning

no code implementations17 Dec 2020 Ashish Kapoor

Commercial aviation is one of the biggest contributors towards climate change.

BIG-bench Machine Learning

Do Adversarially Robust ImageNet Models Transfer Better?

2 code implementations NeurIPS 2020 Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, Aleksander Madry

Typically, better pre-trained models yield better transfer results, suggesting that initial accuracy is a key aspect of transfer learning performance.

Transfer Learning

TartanAir: A Dataset to Push the Limits of Visual SLAM

1 code implementation31 Mar 2020 Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, Sebastian Scherer

We present a challenging dataset, the TartanAir, for robot navigation task and more.


AirSim Drone Racing Lab

2 code implementations12 Mar 2020 Ratnesh Madaan, Nicholas Gyde, Sai Vemprala, Matthew Brown, Keiko Nagami, Tim Taubner, Eric Cristofalo, Davide Scaramuzza, Mac Schwager, Ashish Kapoor

Autonomous drone racing is a challenging research problem at the intersection of computer vision, planning, state estimation, and control.

Benchmarking Optical Flow Estimation

Enhancing Stratospheric Weather Analyses and Forecasts by Deploying Sensors from a Weather Balloon

no code implementations4 Dec 2019 Kiwan Maeng, Iskender Kushan, Brandon Lucia, Ashish Kapoor

We propose a framework to collect stratospheric data by releasing a contrail of tiny sensor devices as a weather balloon ascends.

Modeling Affect-based Intrinsic Rewards for Exploration and Learning

2 code implementations1 Dec 2019 Dean Zadok, Daniel McDuff, Ashish Kapoor

Positive affect has been linked to increased interest, curiosity and satisfaction in human learning.

Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations

2 code implementations16 Sep 2019 Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor

We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors.

Drone navigation Imitation Learning

Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting

2 code implementations NeurIPS 2019 Aditya Grover, Jiaming Song, Alekh Agarwal, Kenneth Tran, Ashish Kapoor, Eric Horvitz, Stefano Ermon

A standard technique to correct this bias is importance sampling, where samples from the model are weighted by the likelihood ratio under model and true distributions.

Data Augmentation

Explorations and Lessons Learned in Building an Autonomous Formula SAE Car from Simulations

4 code implementations15 May 2019 Dean Zadok, Tom Hirshberg, Amir Biran, Kira Radinsky, Ashish Kapoor

This paper describes the exploration and learnings during the process of developing a self-driving algorithm in simulation, followed by deployment on a real car.


Visceral Machines: Reinforcement Learning with Intrinsic Physiological Rewards

no code implementations ICLR 2019 Daniel McDuff, Ashish Kapoor

The human autonomic nervous system has evolved over millions of years and is essential for survival and responding to threats.

Navigate reinforcement-learning +1

Synthetic Examples Improve Generalization for Rare Classes

no code implementations11 Apr 2019 Sara Beery, Yang Liu, Dan Morris, Jim Piavis, Ashish Kapoor, Markus Meister, Neel Joshi, Pietro Perona

The ability to detect and classify rare occurrences in images has important applications - for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars.

Few-Shot Learning Self-Driving Cars

Bias Correction of Learned Generative Models via Likelihood-free Importance Weighting

no code implementations ICLR Workshop DeepGenStruct 2019 Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric Horvitz, Stefano Ermon

A standard technique to correct this bias is by importance weighting samples from the model by the likelihood ratio under the model and true distributions.

Data Augmentation

Identifying Bias in AI using Simulation

no code implementations ICLR 2019 Daniel McDuff, Roger Cheng, Ashish Kapoor

Machine learned models exhibit bias, often because the datasets used to train them are biased.

Face Detection

Visceral Machines: Risk-Aversion in Reinforcement Learning with Intrinsic Physiological Rewards

2 code implementations25 May 2018 Daniel McDuff, Ashish Kapoor

As people learn to navigate the world, autonomic nervous system (e. g., "fight or flight") responses provide intrinsic feedback about the potential consequence of action choices (e. g., becoming nervous when close to a cliff edge or driving fast around a bend.)

Navigate reinforcement-learning +1

Verifying Controllers Against Adversarial Examples with Bayesian Optimization

1 code implementation23 Feb 2018 Shromona Ghosh, Felix Berkenkamp, Gireeja Ranade, Shaz Qadeer, Ashish Kapoor

We specify safety constraints using logic and exploit structure in the problem in order to test the system for adversarial counter examples that violate the safety specifications.

Bayesian Optimization reinforcement-learning +1

Active Learning amidst Logical Knowledge

1 code implementation26 Sep 2017 Emmanouil Antonios Platanios, Ashish Kapoor, Eric Horvitz

Structured prediction is ubiquitous in applications of machine learning such as knowledge extraction and natural language processing.

Active Learning BIG-bench Machine Learning +1

Safety-Aware Algorithms for Adversarial Contextual Bandit

no code implementations ICML 2017 Wen Sun, Debadeepta Dey, Ashish Kapoor

To address this problem, we first study online convex programming in the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.

Decision Making Multi-Armed Bandits

AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles

25 code implementations15 May 2017 Shital Shah, Debadeepta Dey, Chris Lovett, Ashish Kapoor

Developing and testing algorithms for autonomous vehicles in real world is an expensive and time consuming process.

Autonomous Vehicles

Submodular Trajectory Optimization for Aerial 3D Scanning

no code implementations ICCV 2017 Mike Roberts, Debadeepta Dey, Anh Truong, Sudipta Sinha, Shital Shah, Ashish Kapoor, Pat Hanrahan, Neel Joshi

Drones equipped with cameras are emerging as a powerful tool for large-scale aerial 3D scanning, but existing automatic flight planners do not exploit all available information about the scene, and can therefore produce inaccurate and incomplete 3D models.

Trajectory Planning

Learning to Gather Information via Imitation

no code implementations13 Nov 2016 Sanjiban Choudhury, Ashish Kapoor, Gireeja Ranade, Debadeepta Dey

The budgeted information gathering problem - where a robot with a fixed fuel budget is required to maximize the amount of information gathered from the world - appears in practice across a wide range of applications in autonomous exploration and inspection with mobile robots.

Imitation Learning

Risk-Aware Algorithms for Adversarial Contextual Bandits

no code implementations17 Oct 2016 Wen Sun, Debadeepta Dey, Ashish Kapoor

To address this problem, we first study the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.

Multi-Armed Bandits

Quantum Perceptron Models

no code implementations NeurIPS 2016 Nathan Wiebe, Ashish Kapoor, Krysta M. Svore

We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model.

Safe Control under Uncertainty

no code implementations25 Oct 2015 Dorsa Sadigh, Ashish Kapoor

In this paper, we propose a new logic, Probabilistic Signal Temporal Logic (PrSTL), as an expressive language to define the stochastic properties, and enforce probabilistic guarantees on them.

Autonomous Vehicles

Quantum Inspired Training for Boltzmann Machines

no code implementations9 Jul 2015 Nathan Wiebe, Ashish Kapoor, Christopher Granade, Krysta M. Svore

We present an efficient classical algorithm for training deep Boltzmann machines (DBMs) that uses rejection sampling in concert with variational approximations to estimate the gradients of the training objective function.

Inferring and Learning from Neuronal Correspondences

no code implementations23 Jan 2015 Ashish Kapoor, E. Paxon Frady, Stefanie Jegelka, William B. Kristan, Eric Horvitz

We introduce and study methods for inferring and learning from correspondences among neurons.

Decision Making

Quantum Deep Learning

no code implementations10 Dec 2014 Nathan Wiebe, Ashish Kapoor, Krysta M. Svore

In recent years, deep learning has had a profound impact on machine learning and artificial intelligence.

Blind Image Quality Assessment using Semi-supervised Rectifier Networks

no code implementations CVPR 2014 Huixuan Tang, Neel Joshi, Ashish Kapoor

The biggest hurdles to these efforts are: 1) the difficulty of generalizing across diverse types of distortions and 2) collecting the enormity of human scored training data that is needed to learn the measure.

Blind Image Quality Assessment Image Quality Estimation

Riffled Independence for Efficient Inference with Partial Rankings

no code implementations23 Jan 2014 Jonathan Huang, Ashish Kapoor, Carlos Guestrin

Simultaneously addressing all of these challenges i. e., designing a compactly representable model which is amenable to efficient inference and can be learned using partial ranking data is a difficult task, but is necessary if we would like to scale to problems with nontrivial size.

Quantum Algorithms for Nearest-Neighbor Methods for Supervised and Unsupervised Learning

2 code implementations9 Jan 2014 Nathan Wiebe, Ashish Kapoor, Krysta Svore

In the worst case, our quantum algorithms lead to polynomial reductions in query complexity relative to the corresponding classical algorithm.

Quantum Physics

Multilabel Classification using Bayesian Compressed Sensing

no code implementations NeurIPS 2012 Ashish Kapoor, Raajay Viswanathan, Prateek Jain

The two key benefits of the model are that a) it can naturally handle datasets that have missing labels and b) it can also measure uncertainty in prediction.

Active Learning Classification +3

Cannot find the paper you are looking for? You can Submit a new open access paper.