Search Results for author: Debadeepta Dey

Found 32 papers, 9 papers with code

What Makes Convolutional Models Great on Long Sequence Modeling?

1 code implementation17 Oct 2022 Yuhong Li, Tianle Cai, Yi Zhang, Deming Chen, Debadeepta Dey

We focus on the structure of the convolution kernel and identify two critical but intuitive principles enjoyed by S4 that are sufficient to make up an effective global convolutional model: 1) The parameterization of the convolutional kernel needs to be efficient in the sense that the number of parameters should scale sub-linearly with sequence length.

Long-range modeling

Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints

no code implementations6 Oct 2022 Ganesh Jawahar, Subhabrata Mukherjee, Debadeepta Dey, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, Caio Cesar Teodoro Mendes, Gustavo Henrique de Rosa, Shital Shah

In this work, we study the more challenging open-domain setting consisting of low frequency user prompt patterns (or broad prompts, e. g., prompt about 93rd academy awards) and demonstrate the effectiveness of character-based language models.

Inductive Bias

LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models

1 code implementation4 Mar 2022 Mojan Javaheripi, Gustavo H. de Rosa, Subhabrata Mukherjee, Shital Shah, Tomasz L. Religa, Caio C. T. Mendes, Sebastien Bubeck, Farinaz Koushanfar, Debadeepta Dey

Results show that the perplexity of 16-layer GPT-2 and Transformer-XL can be achieved with up to 1. 5x, 2. 5x faster runtime and 1. 2x, 2. 0x lower peak memory utilization.

Decoder Language Modelling +1

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

no code implementations29 Jan 2022 Dongkuan Xu, Subhabrata Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Hassan Awadallah, Jianfeng Gao

Our framework AutoDistil addresses above challenges with the following steps: (a) Incorporates inductive bias and heuristics to partition Transformer search space into K compact sub-spaces (K=3 for typical student sizes of base, small and tiny); (b) Trains one SuperLM for each sub-space using task-agnostic objective (e. g., self-attention distillation) with weight-sharing of students; (c) Lightweight search for the optimal student without re-training.

Inductive Bias Knowledge Distillation +1

Ranking Convolutional Architectures by their Feature Extraction Capabilities

no code implementations29 Sep 2021 Debadeepta Dey, Shital Shah, Sebastien Bubeck

We propose a simple but powerful method which we call FEAR, for ranking architectures in any search space.

Neural Architecture Search

FEAR: A Simple Lightweight Method to Rank Architectures

1 code implementation7 Jun 2021 Debadeepta Dey, Shital Shah, Sebastien Bubeck

We propose a simple but powerful method which we call FEAR, for ranking architectures in any search space.

Neural Architecture Search

Ranking Architectures by Feature Extraction Capabilities

no code implementations ICML Workshop AutoML 2021 Debadeepta Dey, Shital Shah, Sebastien Bubeck

By training different architectures in the search space to the same training or validation error and subsequently comparing the usefulness of the features extracted on the task-dataset of interest by freezing most of the architecture we obtain quick estimates of the relative performance.

Neural Architecture Search

Reparameterized Variational Divergence Minimization for Stable Imitation

no code implementations18 Jun 2020 Dilip Arumugam, Debadeepta Dey, Alekh Agarwal, Asli Celikyilmaz, Elnaz Nouri, Bill Dolan

While recent state-of-the-art results for adversarial imitation-learning algorithms are encouraging, recent works exploring the imitation learning from observation (ILO) setting, where trajectories \textit{only} contain expert observations, have not been met with the same success.

Continuous Control Imitation Learning

A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks

1 code implementation ACL 2020 Angela S. Lin, Sudha Rao, Asli Celikyilmaz, Elnaz Nouri, Chris Brockett, Debadeepta Dey, Bill Dolan

Learning to align these different instruction sets is challenging because: a) different recipes vary in their order of instructions and use of ingredients; and b) video instructions can be noisy and tend to contain far more information than text instructions.


Efficient Forward Architecture Search

2 code implementations NeurIPS 2019 Hanzhang Hu, John Langford, Rich Caruana, Saurajit Mukherjee, Eric Horvitz, Debadeepta Dey

We propose a neural architecture search (NAS) algorithm, Petridish, to iteratively add shortcut connections to existing network layers.

feature selection Neural Architecture Search +1

Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations

no code implementations12 May 2019 Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz

We address the opportunity to maximize the utility of an overall computing system by employing reinforcement learning to guide the configuration of the set of interacting modules that comprise the system.

Decision Making reinforcement-learning +1

Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention

1 code implementation CVPR 2019 Khanh Nguyen, Debadeepta Dey, Chris Brockett, Bill Dolan

We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments.

Imitation Learning Navigate +2

Discovering Blind Spots in Reinforcement Learning

no code implementations23 May 2018 Ramya Ramakrishnan, Ece Kamar, Debadeepta Dey, Julie Shah, Eric Horvitz

Agents trained in simulation may make errors in the real world due to mismatches between training and execution environments.

reinforcement-learning Reinforcement Learning (RL)

Log-DenseNet: How to Sparsify a DenseNet

1 code implementation ICLR 2018 Hanzhang Hu, Debadeepta Dey, Allison Del Giorno, Martial Hebert, J. Andrew Bagnell

Skip connections are increasingly utilized by deep neural networks to improve accuracy and cost-efficiency.

Semantic Segmentation

Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing

no code implementations22 Aug 2017 Hanzhang Hu, Debadeepta Dey, Martial Hebert, J. Andrew Bagnell

Experimentally, the adaptive weights induce more competitive anytime predictions on multiple recognition data-sets and models than non-adaptive approaches including weighing all losses equally.

Safety-Aware Algorithms for Adversarial Contextual Bandit

no code implementations ICML 2017 Wen Sun, Debadeepta Dey, Ashish Kapoor

To address this problem, we first study online convex programming in the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.

Decision Making Multi-Armed Bandits

AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles

25 code implementations15 May 2017 Shital Shah, Debadeepta Dey, Chris Lovett, Ashish Kapoor

Developing and testing algorithms for autonomous vehicles in real world is an expensive and time consuming process.

Autonomous Vehicles

Submodular Trajectory Optimization for Aerial 3D Scanning

no code implementations ICCV 2017 Mike Roberts, Debadeepta Dey, Anh Truong, Sudipta Sinha, Shital Shah, Ashish Kapoor, Pat Hanrahan, Neel Joshi

Drones equipped with cameras are emerging as a powerful tool for large-scale aerial 3D scanning, but existing automatic flight planners do not exploit all available information about the scene, and can therefore produce inaccurate and incomplete 3D models.

Trajectory Planning

Flight Dynamics-based Recovery of a UAV Trajectory using Ground Cameras

no code implementations CVPR 2017 Artem Rozantsev, Sudipta N. Sinha, Debadeepta Dey, Pascal Fua

Our main contribution is a new bundle adjustment procedure which in addition to optimizing the camera poses, regularizes the point trajectory using a prior based on motion dynamics (or specifically flight dynamics).

3D Reconstruction

Learning to Gather Information via Imitation

no code implementations13 Nov 2016 Sanjiban Choudhury, Ashish Kapoor, Gireeja Ranade, Debadeepta Dey

The budgeted information gathering problem - where a robot with a fixed fuel budget is required to maximize the amount of information gathered from the world - appears in practice across a wide range of applications in autonomous exploration and inspection with mobile robots.

Imitation Learning

Risk-Aware Algorithms for Adversarial Contextual Bandits

no code implementations17 Oct 2016 Wen Sun, Debadeepta Dey, Ashish Kapoor

To address this problem, we first study the full information setting where in each round the learner receives an adversarial convex loss and a convex constraint.

Multi-Armed Bandits

Predicting Multiple Structured Visual Interpretations

no code implementations ICCV 2015 Debadeepta Dey, Varun Ramakrishna, Martial Hebert, J. Andrew Bagnell

We present a simple approach for producing a small number of structured visual outputs which have high recall, for a variety of tasks including monocular pose estimation and semantic scene segmentation.

Pose Estimation Scene Segmentation +1

Learning Policies for Contextual Submodular Prediction

no code implementations11 May 2013 Stephane Ross, Jiaji Zhou, Yisong Yue, Debadeepta Dey, J. Andrew Bagnell

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options.

Document Summarization News Recommendation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.