Search Results for author:
Found 34 papers, 20 papers with code
Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting
We describe Honk, an open-source PyTorch reimplementation of convolutional neural networks for keyword spotting that are included as examples in TensorFlow.
Deep Residual Learning for Small-Footprint Keyword Spotting
We explore the application of deep residual learning and dilated convolutions to the keyword spotting task, using the recently-released Google Speech Commands Dataset as our benchmark.
Adaptive Pruning of Neural Language Models for Mobile Devices
Neural language models (NLMs) exist in an accuracy-efficiency tradeoff space where better perplexity typically comes at the cost of greater computation complexity.
JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis
Overall, our robust, cross-device implementation for keyword spotting realizes a new paradigm for serving neural network applications, and one of our slim models reduces latency by 66% with a minimal decrease in accuracy of 4% from 94% to 90%.
Progress and Tradeoffs in Neural Language Models
In recent years, we have witnessed a dramatic shift towards techniques driven by neural networks for a variety of NLP tasks.
FLOPs as a Direct Optimization Objective for Learning Sparse Neural Networks
There exists a plethora of techniques for inducing structured sparsity in parametric models during the optimization process, with the final goal of resource-efficient inference.
Streaming Voice Query Recognition using Causal Convolutional Recurrent Neural Networks
Voice-enabled commercial products are ubiquitous, typically enabled by lightweight on-device keyword spotting (KWS) and full automatic speech recognition (ASR) in the cloud.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
In the natural language processing literature, neural networks are becoming increasingly deeper and complex.
Ranked #60 on
Sentiment Analysis
on SST-2 Binary classification
DocBERT: BERT for Document Classification
We present, to our knowledge, the first application of BERT to document classification.
Ranked #1 on
Document Classification
on Yelp-14
Rethinking Complex Neural Network Architectures for Document Classification
Neural network models for many NLP tasks have grown increasingly complex in recent years, making training and deployment more difficult.
Ranked #2 on
Document Classification
on IMDb-M
Incorporating Contextual and Syntactic Structures Improves Semantic Similarity Modeling
Semantic similarity modeling is central to many NLP problems such as natural language inference and question answering.
Natural Language Generation for Effective Knowledge Distillation
Knowledge distillation can effectively transfer knowledge from BERT, a deep language representation model, to traditional, shallow word embedding-based neural networks, helping them approach or exceed the quality of other heavyweight language representation models.
Honkling: In-Browser Personalization for Ubiquitous Keyword Spotting
Used for simple commands recognition on devices from smart speakers to mobile phones, keyword spotting systems are everywhere.
Explicit Pairwise Word Interaction Modeling Improves Pretrained Transformers for English Semantic Similarity Tasks
In this paper, we hypothesize that introducing an explicit, constrained pairwise word interaction mechanism to pretrained language models improves their effectiveness on semantic similarity tasks.
What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning
We show that only a fourth of the final layers need to be fine-tuned to achieve 90% of the original quality.
Rapidly Bootstrapping a Question Answering Dataset for COVID-19
We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Dataset Challenge.
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications.
Showing Your Work Doesn't Always Work
In natural language processing, a recently popular line of work explores how to best report the experimental results of neural networks.
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT
Fine-tuned variants of BERT are able to achieve state-of-the-art accuracy on many natural language processing tasks, although at significant computational costs.
Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset
We present Covidex, a search engine that exploits the latest neural ranking models to provide information access to the COVID-19 Open Research Dataset curated by the Allen Institute for AI.
Howl: A Deployed, Open-Source Wake Word Detection System
We describe Howl, an open-source wake word detection toolkit with native support for open speech datasets, like Mozilla Common Voice and Google Speech Commands.
Ranked #4 on
Keyword Spotting
on Google Speech Commands
Inserting Information Bottlenecks for Attribution in Transformers
We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers.
BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression
The slow speed of BERT has motivated much research on accelerating its inference, and the early exiting idea has been proposed to make trade-offs between model quality and efficiency.
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing
To fill this void in the literature, we study in this paper selective prediction for NLP, comparing different models and confidence estimators.
Building an Efficiency Pipeline: Commutativity and Cumulativeness of Efficiency Operators for Transformers
There exists a wide variety of efficiency methods for natural language processing (NLP) tasks, such as pruning, distillation, dynamic inference, quantization, etc.
What the DAAM: Interpreting Stable Diffusion Using Cross Attention
Large-scale diffusion neural networks represent a substantial milestone in text-to-image generation, but they remain poorly understood, lacking interpretability analyses.
SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale
In this paper, we explore training and deploying an ASR system in the label-scarce, compute-limited setting.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Less is More: Parameter-Free Text Classification with Gzip
Our method also performs particularly well in few-shot settings where labeled data are too scarce for DNNs to achieve a satisfying accuracy.
Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models
Large language models (LLMs) exhibit positional bias in how they use context, which especially complicates listwise ranking.
Strings from the Library of Babel: Random Sampling as a Strong Baseline for Prompt Optimisation
Recent prompt optimisation approaches use the generative nature of language models to produce prompts -- even rivaling the performance of human-curated prompts.
What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations
We propose a logistic Bradley-Terry probe which predicts word pair preferences of LLMs from the words' hidden vectors.
Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models
However, current works in this direction all depend on the GPT models, making it a single point of failure in scientific reproducibility.
Voice Query Auto Completion
Query auto completion (QAC) is the task of predicting a search engine user’s final query from their intermediate, incomplete query.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
How Does BERT Rerank Passages? An Attribution Analysis with Information Bottlenecks
Fine-tuned pre-trained transformers achieve the state of the art in passage reranking.
