Search Results for author:
Found 34 papers, 20 papers with code
Memory-efficient Transformers via Top-k Attention
Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.
Exploring the limits of decoder-only models trained on public speech recognition corpora
The emergence of industrial-scale speech recognition (ASR) models such as Whisper and USM, trained on 1M hours of weakly labelled and 12M hours of audio only proprietary data respectively, has led to a stronger need for large scale public ASR corpora and competitive open source pipelines.
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
Modeling long-range dependencies across sequences is a longstanding goal in machine learning and has led to architectures, such as state space models, that dramatically outperform Transformers on long sequences.
Data-Driven Computation of Robust Invariant Sets and Gain-Scheduled Controllers for Linear Parameter-Varying Systems
We present a direct data-driven approach to synthesize robust control invariant (RCI) sets and their associated gain-scheduled feedback control laws for linear parameter-varying (LPV) systems subjected to bounded disturbances.
Complete characterization of robust perfect adaptation in biochemical reaction networks
The present results indicate that an RPA property is essentially equivalent to the existence of a "topological invariant", which is an instance of what we call the "Robust Adaptation is Topological"(RAT) principle.
Efficient High-Resolution Template Matching with Vector Quantized Nearest Neighbour Fields
A vector quantization step is introduced before the NN calculation to represent the template with $k$ features, and the filter response over the NNFs is used to compare the template and query distributions over the features.
Direct Data-Driven Computation of Polytopic Robust Control Invariant Sets and State-Feedback Controllers
This paper presents a direct data-driven approach for computing robust control invariant (RCI) sets and their associated state-feedback control laws for linear time-invariant systems affected by bounded disturbances.
Diagonal State Space Augmented Transformers for Speech Recognition
We improve on the popular conformer architecture by replacing the depthwise temporal convolutions with diagonal state space (DSS) models.
Simplifying and Understanding State Space Models with Diagonal Linear RNNs
Sequence models based on linear state spaces (SSMs) have recently emerged as a promising choice of architecture for modeling long range dependencies across various modalities.
A Novel Frame Structure for Cloud-Based Audio-Visual Speech Enhancement in Multimodal Hearing-aids
In this paper, we design a first of its kind transceiver (PHY layer) prototype for cloud-based audio-visual (AV) speech enhancement (SE) complying with high data rate and low latency requirements of future multimodal hearing assistive technology.
Towards Better Guided Attention and Human Knowledge Insertion in Deep Convolutional Neural Networks
In this work, we introduce Multi-Scale Attention Branch Networks (MSABN), which enhance the resolution of the generated attention maps, and improve the performance.
Analyzing Transformers in Embedding Space
In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on.
Long Range Language Modeling via Gated State Spaces
State space models have shown to be effective at modeling long range dependencies, specially on sequence classification tasks.
On the Parameterization and Initialization of Diagonal State Space Models
On the other hand, a recent variant of S4 called DSS showed that restricting the state matrix to be fully diagonal can still preserve the performance of the original model when using a specific initialization based on approximating S4's matrix.
Ranked #9 on
Long-range modeling
on LRA
Diagonal State Spaces are as Effective as Structured State Spaces
Modeling long range dependencies in sequential data is a fundamental step towards attaining human-level performance in many modalities such as text, vision, audio and video.
Ranked #11 on
Long-range modeling
on LRA
SCROLLS: Standardized CompaRison Over Long Language Sequences
NLP benchmarks have largely focused on short texts, such as sentences and paragraphs, even though long texts comprise a considerable amount of natural language in the wild.
Ranked #8 on
Long-range modeling
on SCROLLS
Convergence of regularized particle filters for stochastic reaction networks
In this case, the regularized particle filter (RPF) is preferred to the conventional ones, as the RPF can mitigate sample degeneracy by perturbing particles with artificial noise.
Memory-efficient Transformers via Top-$k$ Attention
Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.
Stochastic filtering for multiscale stochastic reaction networks based on hybrid approximations
As a result, we are able to develop efficient particle filters to solve the filtering problem for the original model by applying particle filters to the reduced model.
Value-aware Approximate Attention
Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length.
Computation of Parameter Dependent Robust Invariant Sets for LPV Models with Guaranteed Performance
This paper presents an iterative algorithm to compute a Robust Control Invariant (RCI) set, along with an invariance-inducing control law, for Linear Parameter-Varying (LPV) systems.
DART: Open-Domain Structured Data Record to Text Generation
Data-to-Text annotations can be a costly process, especially when dealing with tables which are the major source of structured data and contain nontrivial structures.
GMAT: Global Memory Augmentation for Transformers
Moreover, global memory can also be used for sequence compression, by representing a long input sequence with the memory representations only.
Injecting Numerical Reasoning Skills into Language Models
In this work, we show that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup.
Ranked #9 on
Question Answering
on DROP Test
Braitenberg Vehicles as Developmental Neurosimulation
As is standard among models of artificial and biological neural networks, an analogue of the fully mature brain is presented as a blank slate.
Break It Down: A Question Understanding Benchmark
Understanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer.
A hidden integral structure endows Absolute Concentration Robust systems with resilience to dynamical concentration disturbances
These species behave in a stable, predictable way, in the sense that their expression is robust with respect to sudden changes in the species concentration, regardless the new positive equilibrium reached by the system.
HeartFit: An Accurate Platform for Heart Murmur Diagnosis Utilizing Deep Learning
Using a deep learning network architecture, the database classifies the audio and returns the diagnosis to the user.
CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection
As machine learning and cybersecurity continue to explode in the context of the digital ecosystem, the complexity of cybersecurity data combined with complicated and evasive machine learning algorithms leads to vast difficulties in designing an end to end system for intelligent, automatic anomaly classification.
User-Interactive Machine Learning Model for Identifying Structural Relationships of Code Features
Based off of the results of the user study, user interaction with the rules interface corrects feature relationships missed or mistaken by the automated process, enhancing autocomplete accuracy and developer productivity.
AquaSight: Automatic Water Impurity Detection Utilizing Convolutional Neural Networks
According to the United Nations World Water Assessment Programme, every day, 2 million tons of sewage and industrial and agricultural waste are discharged into the worlds water.
StrokeSave: A Novel, High-Performance Mobile Application for Stroke Diagnosis using Deep Learning and Computer Vision
StrokeSave is a platform for users to self-diagnose for prevalence to stroke.
Dilated Convolutions for Modeling Long-Distance Genomic Dependencies
We consider the task of detecting regulatory elements in the human genome directly from raw DNA.
A Comparative Analysis of Tensor Decomposition Models Using Hyper Spectral Image
Our approach deals with implementing three decomposition models LMLRA, BTD and CPD to the sample data for choosing the best decomposition of the data set.
