Search Results for author: Raphael Tang

Found 32 papers, 19 papers with code

What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations

1 code implementation30 Nov 2023 Raphael Tang, Xinyu Zhang, Jimmy Lin, Ferhan Ture

We propose a logistic Bradley-Terry probe which predicts word pair preferences of LLMs from the words' hidden vectors.

Language Modelling

Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models

1 code implementation11 Oct 2023 Raphael Tang, Xinyu Zhang, Xueguang Ma, Jimmy Lin, Ferhan Ture

Large language models (LLMs) exhibit positional bias in how they use context, which especially complicates listwise ranking.

Less is More: Parameter-Free Text Classification with Gzip

no code implementations19 Dec 2022 Zhiying Jiang, Matthew Y. R. Yang, Mikhail Tsirlin, Raphael Tang, Jimmy Lin

Our method also performs particularly well in few-shot settings where labeled data are too scarce for DNNs to achieve a satisfying accuracy.

text-classification Text Classification

What the DAAM: Interpreting Stable Diffusion Using Cross Attention

1 code implementation10 Oct 2022 Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, Ferhan Ture

Large-scale diffusion neural networks represent a substantial milestone in text-to-image generation, but they remain poorly understood, lacking interpretability analyses.

Denoising Descriptive +2

Building an Efficiency Pipeline: Commutativity and Cumulativeness of Efficiency Operators for Transformers

no code implementations31 Jul 2022 Ji Xin, Raphael Tang, Zhiying Jiang, YaoLiang Yu, Jimmy Lin

There exists a wide variety of efficiency methods for natural language processing (NLP) tasks, such as pruning, distillation, dynamic inference, quantization, etc.


The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing

1 code implementation ACL 2021 Ji Xin, Raphael Tang, YaoLiang Yu, Jimmy Lin

To fill this void in the literature, we study in this paper selective prediction for NLP, comparing different models and confidence estimators.

BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression

1 code implementation EACL 2021 Ji Xin, Raphael Tang, YaoLiang Yu, Jimmy Lin

The slow speed of BERT has motivated much research on accelerating its inference, and the early exiting idea has been proposed to make trade-offs between model quality and efficiency.


Inserting Information Bottlenecks for Attribution in Transformers

1 code implementation Findings of the Association for Computational Linguistics 2020 Zhiying Jiang, Raphael Tang, Ji Xin, Jimmy Lin

We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers.

Howl: A Deployed, Open-Source Wake Word Detection System

2 code implementations EMNLP (NLPOSS) 2020 Raphael Tang, Jaejun Lee, Afsaneh Razi, Julia Cambre, Ian Bicking, Jofish Kaye, Jimmy Lin

We describe Howl, an open-source wake word detection toolkit with native support for open speech datasets, like Mozilla Common Voice and Google Speech Commands.

Keyword Spotting

Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset

1 code implementation EMNLP (sdp) 2020 Edwin Zhang, Nikhil Gupta, Raphael Tang, Xiao Han, Ronak Pradeep, Kuang Lu, Yue Zhang, Rodrigo Nogueira, Kyunghyun Cho, Hui Fang, Jimmy Lin

We present Covidex, a search engine that exploits the latest neural ranking models to provide information access to the COVID-19 Open Research Dataset curated by the Allen Institute for AI.

Showing Your Work Doesn't Always Work

1 code implementation ACL 2020 Raphael Tang, Jaejun Lee, Ji Xin, Xinyu Liu, Yao-Liang Yu, Jimmy Lin

In natural language processing, a recently popular line of work explores how to best report the experimental results of neural networks.

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

3 code implementations ACL 2020 Ji Xin, Raphael Tang, Jaejun Lee, Yao-Liang Yu, Jimmy Lin

Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications.

Rapidly Bootstrapping a Question Answering Dataset for COVID-19

1 code implementation23 Apr 2020 Raphael Tang, Rodrigo Nogueira, Edwin Zhang, Nikhil Gupta, Phuong Cam, Kyunghyun Cho, Jimmy Lin

We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Dataset Challenge.

Question Answering

What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning

no code implementations8 Nov 2019 Jaejun Lee, Raphael Tang, Jimmy Lin

We show that only a fourth of the final layers need to be fine-tuned to achieve 90% of the original quality.

Linguistic Acceptability Natural Language Inference +3

Explicit Pairwise Word Interaction Modeling Improves Pretrained Transformers for English Semantic Similarity Tasks

no code implementations7 Nov 2019 Yinan Zhang, Raphael Tang, Jimmy Lin

In this paper, we hypothesize that introducing an explicit, constrained pairwise word interaction mechanism to pretrained language models improves their effectiveness on semantic similarity tasks.

Semantic Similarity Semantic Textual Similarity

Honkling: In-Browser Personalization for Ubiquitous Keyword Spotting

1 code implementation IJCNLP 2019 Jaejun Lee, Raphael Tang, Jimmy Lin

Used for simple commands recognition on devices from smart speakers to mobile phones, keyword spotting systems are everywhere.

Keyword Spotting

Natural Language Generation for Effective Knowledge Distillation

1 code implementation WS 2019 Raphael Tang, Yao Lu, Jimmy Lin

Knowledge distillation can effectively transfer knowledge from BERT, a deep language representation model, to traditional, shallow word embedding-based neural networks, helping them approach or exceed the quality of other heavyweight language representation models.

Knowledge Distillation Linguistic Acceptability +5

Streaming Voice Query Recognition using Causal Convolutional Recurrent Neural Networks

no code implementations19 Dec 2018 Raphael Tang, Gefei Yang, Hong Wei, Yajie Mao, Ferhan Ture, Jimmy Lin

Voice-enabled commercial products are ubiquitous, typically enabled by lightweight on-device keyword spotting (KWS) and full automatic speech recognition (ASR) in the cloud.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

FLOPs as a Direct Optimization Objective for Learning Sparse Neural Networks

no code implementations NIPS Workshop CDNNRIA 2018 Raphael Tang, Ashutosh Adhikari, Jimmy Lin

There exists a plethora of techniques for inducing structured sparsity in parametric models during the optimization process, with the final goal of resource-efficient inference.

Image Classification Model Compression

Progress and Tradeoffs in Neural Language Models

no code implementations2 Nov 2018 Raphael Tang, Jimmy Lin

In recent years, we have witnessed a dramatic shift towards techniques driven by neural networks for a variety of NLP tasks.

Language Modelling

JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis

1 code implementation30 Oct 2018 Jaejun Lee, Raphael Tang, Jimmy Lin

Overall, our robust, cross-device implementation for keyword spotting realizes a new paradigm for serving neural network applications, and one of our slim models reduces latency by 66% with a minimal decrease in accuracy of 4% from 94% to 90%.

Keyword Spotting Model Compression

Adaptive Pruning of Neural Language Models for Mobile Devices

no code implementations ICLR 2019 Raphael Tang, Jimmy Lin

Neural language models (NLMs) exist in an accuracy-efficiency tradeoff space where better perplexity typically comes at the cost of greater computation complexity.

Deep Residual Learning for Small-Footprint Keyword Spotting

4 code implementations28 Oct 2017 Raphael Tang, Jimmy Lin

We explore the application of deep residual learning and dilated convolutions to the keyword spotting task, using the recently-released Google Speech Commands Dataset as our benchmark.

Small-Footprint Keyword Spotting

Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting

4 code implementations18 Oct 2017 Raphael Tang, Jimmy Lin

We describe Honk, an open-source PyTorch reimplementation of convolutional neural networks for keyword spotting that are included as examples in TensorFlow.

Keyword Spotting speech-recognition +1

Cannot find the paper you are looking for? You can Submit a new open access paper.