Search Results for author: Ankit Gupta

Found 34 papers, 21 papers with code

Memory-efficient Transformers via Top-k Attention

1 code implementation EMNLP (sustainlp) 2021 Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonathan Berant

Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.

Exploring the limits of decoder-only models trained on public speech recognition corpora

no code implementations31 Jan 2024 Ankit Gupta, George Saon, Brian Kingsbury

The emergence of industrial-scale speech recognition (ASR) models such as Whisper and USM, trained on 1M hours of weakly labelled and 12M hours of audio only proprietary data respectively, has led to a stronger need for large scale public ASR corpora and competitive open source pipelines.

Decoder speech-recognition +1

Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors

1 code implementation4 Oct 2023 Ido Amos, Jonathan Berant, Ankit Gupta

Modeling long-range dependencies across sequences is a longstanding goal in machine learning and has led to architectures, such as state space models, that dramatically outperform Transformers on long sequences.

Denoising

Data-Driven Computation of Robust Invariant Sets and Gain-Scheduled Controllers for Linear Parameter-Varying Systems

no code implementations4 Sep 2023 Manas Mejari, Ankit Gupta, Dario Piga

We present a direct data-driven approach to synthesize robust control invariant (RCI) sets and their associated gain-scheduled feedback control laws for linear parameter-varying (LPV) systems subjected to bounded disturbances.

Scheduling

Complete characterization of robust perfect adaptation in biochemical reaction networks

1 code implementation14 Jul 2023 Yuji Hirono, Ankit Gupta, Mustafa Khammash

The present results indicate that an RPA property is essentially equivalent to the existence of a "topological invariant", which is an instance of what we call the "Robust Adaptation is Topological"(RAT) principle.

Efficient High-Resolution Template Matching with Vector Quantized Nearest Neighbour Fields

1 code implementation26 Jun 2023 Ankit Gupta, Ida-Maria Sintorn

A vector quantization step is introduced before the NN calculation to represent the template with $k$ features, and the filter response over the NNFs is used to compare the template and query distributions over the features.

Image Registration object-detection +4

Direct Data-Driven Computation of Polytopic Robust Control Invariant Sets and State-Feedback Controllers

no code implementations31 Mar 2023 Manas Mejari, Ankit Gupta

This paper presents a direct data-driven approach for computing robust control invariant (RCI) sets and their associated state-feedback control laws for linear time-invariant systems affected by bounded disturbances.

Diagonal State Space Augmented Transformers for Speech Recognition

no code implementations27 Feb 2023 George Saon, Ankit Gupta, Xiaodong Cui

We improve on the popular conformer architecture by replacing the depthwise temporal convolutions with diagonal state space (DSS) models.

speech-recognition Speech Recognition

Simplifying and Understanding State Space Models with Diagonal Linear RNNs

1 code implementation1 Dec 2022 Ankit Gupta, Harsh Mehta, Jonathan Berant

Sequence models based on linear state spaces (SSMs) have recently emerged as a promising choice of architecture for modeling long range dependencies across various modalities.

8k

A Novel Frame Structure for Cloud-Based Audio-Visual Speech Enhancement in Multimodal Hearing-aids

no code implementations24 Oct 2022 Abhijeet Bishnu, Ankit Gupta, Mandar Gogate, Kia Dashtipour, Ahsan Adeel, Amir Hussain, Mathini Sellathurai, Tharmalingam Ratnarajah

In this paper, we design a first of its kind transceiver (PHY layer) prototype for cloud-based audio-visual (AV) speech enhancement (SE) complying with high data rate and low latency requirements of future multimodal hearing assistive technology.

Lip Reading Speech Enhancement

Towards Better Guided Attention and Human Knowledge Insertion in Deep Convolutional Neural Networks

no code implementations20 Oct 2022 Ankit Gupta, Ida-Maria Sintorn

In this work, we introduce Multi-Scale Attention Branch Networks (MSABN), which enhance the resolution of the generated attention maps, and improve the performance.

Data Augmentation

Analyzing Transformers in Embedding Space

1 code implementation6 Sep 2022 Guy Dar, Mor Geva, Ankit Gupta, Jonathan Berant

In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on.

Long Range Language Modeling via Gated State Spaces

1 code implementation27 Jun 2022 Harsh Mehta, Ankit Gupta, Ashok Cutkosky, Behnam Neyshabur

State space models have shown to be effective at modeling long range dependencies, specially on sequence classification tasks.

Language Modelling Zero-shot Generalization

On the Parameterization and Initialization of Diagonal State Space Models

2 code implementations23 Jun 2022 Albert Gu, Ankit Gupta, Karan Goel, Christopher Ré

On the other hand, a recent variant of S4 called DSS showed that restricting the state matrix to be fully diagonal can still preserve the performance of the original model when using a specific initialization based on approximating S4's matrix.

Long-range modeling Time Series Analysis

Diagonal State Spaces are as Effective as Structured State Spaces

2 code implementations27 Mar 2022 Ankit Gupta, Albert Gu, Jonathan Berant

Modeling long range dependencies in sequential data is a fundamental step towards attaining human-level performance in many modalities such as text, vision, audio and video.

Long-range modeling

SCROLLS: Standardized CompaRison Over Long Language Sequences

2 code implementations10 Jan 2022 Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer Levy

NLP benchmarks have largely focused on short texts, such as sentences and paragraphs, even though long texts comprise a considerable amount of natural language in the wild.

Decoder Long-range modeling +2

Convergence of regularized particle filters for stochastic reaction networks

1 code implementation14 Oct 2021 Zhou Fang, Ankit Gupta, Mustafa Khammash

In this case, the regularized particle filter (RPF) is preferred to the conventional ones, as the RPF can mitigate sample degeneracy by perturbing particles with artificial noise.

Memory-efficient Transformers via Top-$k$ Attention

2 code implementations13 Jun 2021 Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonathan Berant

Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.

Stochastic filtering for multiscale stochastic reaction networks based on hybrid approximations

1 code implementation6 Jun 2021 Zhou Fang, Ankit Gupta, Mustafa Khammash

As a result, we are able to develop efficient particle filters to solve the filtering problem for the original model by applying particle filters to the reduced model.

Computational Efficiency

Value-aware Approximate Attention

1 code implementation EMNLP 2021 Ankit Gupta, Jonathan Berant

Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.

Language Modelling

Computation of Parameter Dependent Robust Invariant Sets for LPV Models with Guaranteed Performance

no code implementations21 Sep 2020 Ankit Gupta, Manas Mejari, Paolo Falcone, Dario Piga

This paper presents an iterative algorithm to compute a Robust Control Invariant (RCI) set, along with an invariance-inducing control law, for Linear Parameter-Varying (LPV) systems.

Scheduling

GMAT: Global Memory Augmentation for Transformers

1 code implementation5 Jun 2020 Ankit Gupta, Jonathan Berant

Moreover, global memory can also be used for sequence compression, by representing a long input sequence with the memory representations only.

Language Modelling Masked Language Modeling +1

Injecting Numerical Reasoning Skills into Language Models

2 code implementations ACL 2020 Mor Geva, Ankit Gupta, Jonathan Berant

In this work, we show that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup.

Data Augmentation Decoder +3

Braitenberg Vehicles as Developmental Neurosimulation

1 code implementation28 Feb 2020 Stefan Dvoretskii, Ziyi Gong, Ankit Gupta, Jesse Parent, Bradly Alicea

As is standard among models of artificial and biological neural networks, an analogue of the fully mature brain is presented as a blank slate.

Developmental Learning

A hidden integral structure endows Absolute Concentration Robust systems with resilience to dynamical concentration disturbances

no code implementations12 Oct 2019 Daniele Cappelletti, Ankit Gupta, Mustafa Khammash

These species behave in a stable, predictable way, in the sense that their expression is robust with respect to sudden changes in the species concentration, regardless the new positive equilibrium reached by the system.

HeartFit: An Accurate Platform for Heart Murmur Diagnosis Utilizing Deep Learning

no code implementations24 Jul 2019 Ankit Gupta, George Tang, Sylesh Suresh

Using a deep learning network architecture, the database classifies the audio and returns the diagnosis to the user.

CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection

no code implementations23 Jul 2019 Ayush Hariharan, Ankit Gupta, Trisha Pal

As machine learning and cybersecurity continue to explode in the context of the digital ecosystem, the complexity of cybersecurity data combined with complicated and evasive machine learning algorithms leads to vast difficulties in designing an end to end system for intelligent, automatic anomaly classification.

Anomaly Classification Anomaly Detection +2

User-Interactive Machine Learning Model for Identifying Structural Relationships of Code Features

no code implementations18 Jul 2019 Ankit Gupta, Kartik Chugh, Andrea Solis, Thomas LaToza

Based off of the results of the user study, user interaction with the rules interface corrects feature relationships missed or mistaken by the automated process, enhancing autocomplete accuracy and developer productivity.

BIG-bench Machine Learning Code Completion +1

AquaSight: Automatic Water Impurity Detection Utilizing Convolutional Neural Networks

1 code implementation17 Jul 2019 Ankit Gupta, Elliott Ruebush

According to the United Nations World Water Assessment Programme, every day, 2 million tons of sewage and industrial and agricultural waste are discharged into the worlds water.

Dilated Convolutions for Modeling Long-Distance Genomic Dependencies

1 code implementation3 Oct 2017 Ankit Gupta, Alexander M. Rush

We consider the task of detecting regulatory elements in the human genome directly from raw DNA.

A Comparative Analysis of Tensor Decomposition Models Using Hyper Spectral Image

no code implementations23 Mar 2015 Ankit Gupta, Ashish Oberoi

Our approach deals with implementing three decomposition models LMLRA, BTD and CPD to the sample data for choosing the best decomposition of the data set.

Tensor Decomposition

Cannot find the paper you are looking for? You can Submit a new open access paper.