Search Results for author: Yonatan Belinkov

Found 62 papers, 29 papers with code

On the Pitfalls of Analyzing Individual Neurons in Language Models

no code implementations14 Oct 2021 Omer Antverg, Yonatan Belinkov

Among these, the common approach is to use an external probe to rank neurons according to their relevance to some linguistic attribute, and to evaluate the obtained ranking using the same probe that produced it.

Variance Pruning: Pruning Language Models via Temporal Neuron Variance

no code implementations29 Sep 2021 Berry Weinstein, Yonatan Belinkov

As language models become larger, different pruning methods have been proposed to reduce model size.

Natural Language Understanding

A Generative Approach for Mitigating Structural Biases in Natural Language Inference

1 code implementation31 Aug 2021 Dimion Asael, Zachary Ziegler, Yonatan Belinkov

Many natural language inference (NLI) datasets contain biases that allow models to perform well by only using a biased subset of the input, without considering the remainder features.

Natural Language Inference

Variational Information Bottleneck for Effective Low-Resource Fine-Tuning

1 code implementation ICLR 2021 Rabeeh Karimi Mahabadi, Yonatan Belinkov, James Henderson

Moreover, we show that our VIB model finds sentence representations that are more robust to biases in natural language inference datasets, and thereby obtains better generalization to out-of-domain datasets.

Natural Language Inference Transfer Learning

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

1 code implementation ACL 2021 Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, Yonatan Belinkov

Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts.

Supervising Model Attention with Human Explanations for Robust Natural Language Inference

no code implementations16 Apr 2021 Joe Stacey, Yonatan Belinkov, Marek Rei

Natural Language Inference (NLI) models are known to learn from biases and artefacts within their training data, impacting how well they generalise to other unseen datasets.

Natural Language Inference

Probing Classifiers: Promises, Shortcomings, and Advances

no code implementations24 Feb 2021 Yonatan Belinkov

Probing classifiers have emerged as one of the prominent methodologies for interpreting and analyzing deep neural network models of natural language processing.

Learning from others' mistakes: Avoiding dataset biases without modeling them

no code implementations ICLR 2021 Victor Sanh, Thomas Wolf, Yonatan Belinkov, Alexander M. Rush

State-of-the-art natural language processing (NLP) models often learn to model dataset biases and surface form correlations instead of features that target the intended underlying task.

Similarity Analysis of Self-Supervised Speech Representations

no code implementations22 Oct 2020 Yu-An Chung, Yonatan Belinkov, James Glass

We also design probing tasks to study the correlation between the models' pre-training loss and the amount of specific speech information contained in their learned representations.

Representation Learning

Analyzing Individual Neurons in Pre-trained Language Models

1 code implementation EMNLP 2020 Nadir Durrani, Hassan Sajjad, Fahim Dalvi, Yonatan Belinkov

We found small subsets of neurons to predict linguistic tasks, with lower level tasks (such as morphology) localized in fewer neurons, compared to higher level task of predicting syntax.

Interpretability and Analysis in Neural NLP

no code implementations ACL 2020 Yonatan Belinkov, Sebastian Gehrmann, Ellie Pavlick

While deep learning has transformed the natural language processing (NLP) field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior.

Probing Neural Dialog Models for Conversational Understanding

1 code implementation WS 2020 Abdelrhman Saleh, Tovly Deutsch, Stephen Casper, Yonatan Belinkov, Stuart Shieber

The predominant approach to open-domain dialog generation relies on end-to-end training of neural models on chat datasets.

Open-Domain Dialog

The Sensitivity of Language Models and Humans to Winograd Schema Perturbations

1 code implementation ACL 2020 Mostafa Abdou, Vinit Ravishankar, Maria Barrett, Yonatan Belinkov, Desmond Elliott, Anders Søgaard

Large-scale pretrained language models are the major driving force behind recent improvements in performance on the Winograd Schema Challenge, a widely employed test of common sense reasoning ability.

Common Sense Reasoning

Similarity Analysis of Contextual Word Representation Models

1 code implementation ACL 2020 John M. Wu, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, James Glass

We use existing and novel similarity measures that aim to gauge the level of localization of information in the deep models, and facilitate the investigation of which design factors affect model similarity, without requiring any external linguistic annotation.

Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?

no code implementations EACL 2021 Abhilasha Ravichander, Yonatan Belinkov, Eduard Hovy

Although neural models have achieved impressive results on several NLP benchmarks, little is understood about the mechanisms they use to perform language tasks.

Natural Language Inference Word Embeddings

Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias

1 code implementation26 Apr 2020 Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang, Yaron Singer, Stuart Shieber

Common methods for interpreting neural models in natural language processing typically examine either their structure or their behavior, but not both.

Analyzing Redundancy in Pretrained Transformer Models

1 code implementation EMNLP 2020 Fahim Dalvi, Hassan Sajjad, Nadir Durrani, Yonatan Belinkov

Transformer-based deep NLP models are trained using hundreds of millions of parameters, limiting their applicability in computationally constrained environments.

Feature Selection Transfer Learning

Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages

2 code implementations8 Nov 2019 Mirac Suzgun, Sebastian Gehrmann, Yonatan Belinkov, Stuart M. Shieber

We introduce three memory-augmented Recurrent Neural Networks (MARNNs) and explore their capabilities on a series of simple language modeling tasks whose solutions require stack-based mechanisms.

Language Modelling

A Constructive Prediction of the Generalization Error Across Scales

no code implementations ICLR 2020 Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, Nir Shavit

In this work, we present a functional form which approximates well the generalization error in practice.

End-to-End Bias Mitigation by Modelling Biases in Corpora

2 code implementations ACL 2020 Rabeeh Karimi Mahabadi, Yonatan Belinkov, James Henderson

We experiment on large-scale natural language inference and fact verification benchmarks, evaluating on out-of-domain datasets that are specifically designed to assess the robustness of models against known biases in the training data.

Fact Verification Natural Language Inference +1

Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference

1 code implementation ACL 2019 Yonatan Belinkov, Adam Poliak, Stuart M. Shieber, Benjamin Van Durme, Alexander M. Rush

In contrast to standard approaches to NLI, our methods predict the probability of a premise given a hypothesis and NLI label, discouraging models from ignoring the premise.

Natural Language Inference

Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition

1 code implementation9 Jul 2019 Yonatan Belinkov, Ahmed Ali, James Glass

End-to-end neural network systems for automatic speech recognition (ASR) are trained from acoustic features to text transcriptions.

Speech Recognition

Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects

1 code implementation NAACL 2019 Gabriel Grand, Yonatan Belinkov

Visual question answering (VQA) models have been shown to over-rely on linguistic biases in VQA datasets, answering questions "blindly" without considering visual context.

Question Answering Visual Question Answering

LSTM Networks Can Perform Dynamic Counting

no code implementations WS 2019 Mirac Suzgun, Sebastian Gehrmann, Yonatan Belinkov, Stuart M. Shieber

In this paper, we systematically assess the ability of standard recurrent networks to perform dynamic counting and to encode hierarchical representations.

Analyzing the Structure of Attention in a Transformer Language Model

no code implementations WS 2019 Jesse Vig, Yonatan Belinkov

The Transformer is a fully attention-based alternative to recurrent networks that has achieved state-of-the-art results across a range of NLP tasks.

Language Modelling

Improving Neural Language Models by Segmenting, Attending, and Predicting the Future

1 code implementation ACL 2019 Hongyin Luo, Lan Jiang, Yonatan Belinkov, James Glass

In this work, we propose a method that improves language modeling by learning to align the given context and the following phrase.

Language Modelling

One Size Does Not Fit All: Comparing NMT Representations of Different Granularities

no code implementations NAACL 2019 Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Yonatan Belinkov, Preslav Nakov

Recent work has shown that contextualized word representations derived from neural machine translation are a viable alternative to such from simple word predictions tasks.

Machine Translation Translation

Linguistic Knowledge and Transferability of Contextual Representations

no code implementations NAACL 2019 Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith

Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferable features of language.

Language Modelling

Character-based Surprisal as a Model of Reading Difficulty in the Presence of Error

no code implementations2 Feb 2019 Michael Hahn, Frank Keller, Yonatan Bisk, Yonatan Belinkov

Also, transpositions are more difficult than misspellings, and a high error rate increases difficulty for all words, including correct ones.

NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks

2 code implementations21 Dec 2018 Fahim Dalvi, Avery Nortonsmith, D. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, James Glass

We present a toolkit to facilitate the interpretation and understanding of neural network models.

Analysis Methods in Neural Language Processing: A Survey

no code implementations TACL 2019 Yonatan Belinkov, James Glass

The field of natural language processing has seen impressive progress in recent years, with neural network models replacing many of the traditional systems.

What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models

1 code implementation21 Dec 2018 Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Anthony Bau, James Glass

We further present a comprehensive analysis of neurons with the aim to address the following questions: i) how localized or distributed are different linguistic properties in the models?

Language Modelling Machine Translation

On Evaluating the Generalization of LSTM Models in Formal Languages

1 code implementation WS 2019 Mirac Suzgun, Yonatan Belinkov, Stuart M. Shieber

Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing.

Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus

1 code implementation11 Sep 2018 Yonatan Belinkov, Alexander Magidow, Alberto Barrón-Cedeño, Avi Shmidman, Maxim Romanov

Arabic is a widely-spoken language with a long and rich history, but existing corpora and language technology focus mostly on modern Arabic and its varieties.

On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference

1 code implementation NAACL 2018 Adam Poliak, Yonatan Belinkov, James Glass, Benjamin Van Durme

We propose a process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena.

Machine Translation Natural Language Inference +1

Synthetic and Natural Noise Both Break Neural Machine Translation

3 code implementations ICLR 2018 Yonatan Belinkov, Yonatan Bisk

Character-based neural machine translation (NMT) models alleviate out-of-vocabulary issues, learn morphology, and move us closer to completely end-to-end translation systems.

Machine Translation Translation

Understanding and Improving Morphological Learning in the Neural Machine Translation Decoder

no code implementations IJCNLP 2017 Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Stephan Vogel

End-to-end training makes the neural machine translation (NMT) architecture simpler, yet elegant compared to traditional statistical machine translation (SMT).

Machine Translation Multi-Task Learning +1

Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems

1 code implementation NeurIPS 2017 Yonatan Belinkov, James Glass

In this work, we analyze the speech representations learned by a deep end-to-end model that is based on convolutional and recurrent layers, and trained with a connectionist temporal classification (CTC) loss.

General Classification Speech Recognition

Neural Machine Translation Training in a Multi-Domain Scenario

no code implementations29 Aug 2017 Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Yonatan Belinkov, Stephan Vogel

Model stacking works best when training begins with the furthest out-of-domain data and the model is incrementally fine-tuned with the next furthest domain and so on.

Machine Translation Translation

Shamela: A Large-Scale Historical Arabic Corpus

no code implementations WS 2016 Yonatan Belinkov, Alexander Magidow, Maxim Romanov, Avi Shmidman, Moshe Koppel

Arabic is a widely-spoken language with a rich and long history spanning more than fourteen centuries.

Large-Scale Machine Translation between Arabic and Hebrew: Available Corpora and Initial Results

no code implementations25 Sep 2016 Yonatan Belinkov, James Glass

Machine translation between Arabic and Hebrew has so far been limited by a lack of parallel corpora, despite the political and cultural importance of this language pair.

Machine Translation Translation

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks

3 code implementations15 Aug 2016 Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, Yoav Goldberg

The analysis sheds light on the relative strengths of different sentence embedding methods with respect to these low level prediction tasks, and on the effect of the encoded vector's dimensionality on the resulting representations.

Sentence Embedding

Cannot find the paper you are looking for? You can Submit a new open access paper.