Search Results for author: Nadir Durrani

Found 66 papers, 15 papers with code

Scaling up Discovery of Latent Concepts in Deep NLP Models

1 code implementation20 Aug 2023 Majd Hawasly, Fahim Dalvi, Nadir Durrani

Despite the revolution caused by deep NLP models, they remain black boxes, necessitating research to understand their decision-making processes.

Clustering Decision Making

NeuroX Library for Neuron Analysis of Deep NLP Models

1 code implementation26 May 2023 Fahim Dalvi, Hassan Sajjad, Nadir Durrani

The Python toolkit is available at https://www. github. com/fdalvi/NeuroX.

Domain Adaptation

Can LLMs facilitate interpretation of pre-trained language models?

no code implementations22 May 2023 Basel Mousi, Nadir Durrani, Fahim Dalvi

We propose using a large language model, ChatGPT, as an annotator to enable fine-grained interpretation analysis of pre-trained language models.

Language Modelling Large Language Model

NxPlain: Web-based Tool for Discovery of Latent Concepts

no code implementations6 Mar 2023 Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Tamim Jaban, Musab Husaini, Ummar Abbas

NxPlain discovers latent concepts learned in a deep NLP model, provides an interpretation of the knowledge learned in the model, and explains its predictions based on the used concepts.

Fairness Sentence

ConceptX: A Framework for Latent Concept Analysis

no code implementations12 Nov 2022 Firoj Alam, Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Abdul Rafae Khan, Jia Xu

We use an unsupervised method to discover concepts learned in these models and enable a graphical interface for humans to generate explanations for the concepts.

Post-hoc analysis of Arabic transformer models

no code implementations18 Oct 2022 Ahmed Abdelali, Nadir Durrani, Fahim Dalvi, Hassan Sajjad

Given the success of pre-trained language models, many transformer models trained on Arabic and its dialects have surfaced.

Morphological Tagging

Discovering Salient Neurons in Deep NLP Models

no code implementations27 Jun 2022 Nadir Durrani, Fahim Dalvi, Hassan Sajjad

Our data-driven, quantitative analysis illuminates interesting findings: (i) we found small subsets of neurons that can predict different linguistic tasks, ii) with neurons capturing basic lexical information (such as suffixation) localized in lower most layers, iii) while those learning complex concepts (such as syntactic role) predominantly in middle and higher layers, iii) that salient linguistic neurons are relocated from higher to lower layers during transfer learning, as the network preserve the higher layers for task specific information, iv) we found interesting differences across pre-trained models, with respect to how linguistic information is preserved within, and v) we found that concept exhibit similar neuron distribution across different languages in the multilingual transformer models.

Transfer Learning

Analyzing Encoded Concepts in Transformer Language Models

1 code implementation NAACL 2022 Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Firoj Alam, Abdul Rafae Khan, Jia Xu

We propose a novel framework ConceptX, to analyze how latent concepts are encoded in representations learned within pre-trained language models.

Clustering

NatiQ: An End-to-end Text-to-Speech System for Arabic

no code implementations15 Jun 2022 Ahmed Abdelali, Nadir Durrani, Cenk Demiroglu, Fahim Dalvi, Hamdy Mubarak, Kareem Darwish

We concatenated Tacotron1 with the WaveRNN vocoder, Tacotron2 with the WaveGlow vocoder and ESPnet transformer with the parallel wavegan vocoder to synthesize waveforms from the spectrograms.

Discovering Latent Concepts Learned in BERT

no code implementations ICLR 2022 Fahim Dalvi, Abdul Rafae Khan, Firoj Alam, Nadir Durrani, Jia Xu, Hassan Sajjad

We address this limitation by discovering and analyzing latent concepts learned in neural network models in an unsupervised fashion and provide interpretations from the model's perspective.

Novel Concepts POS

Neuron-level Interpretation of Deep NLP Models: A Survey

no code implementations30 Aug 2021 Hassan Sajjad, Nadir Durrani, Fahim Dalvi

The proliferation of deep neural networks in various domains has seen an increased need for interpretability of these models.

Domain Adaptation

How transfer learning impacts linguistic knowledge in deep NLP models?

no code implementations Findings (ACL) 2021 Nadir Durrani, Hassan Sajjad, Fahim Dalvi

The pattern varies across architectures, with BERT retaining linguistic information relatively deeper in the network compared to RoBERTa and XLNet, where it is predominantly delegated to the lower layers.

Transfer Learning

Fine-grained Interpretation and Causation Analysis in Deep NLP Models

no code implementations NAACL 2021 Hassan Sajjad, Narine Kokhlikyan, Fahim Dalvi, Nadir Durrani

This paper is a write-up for the tutorial on "Fine-grained Interpretation and Causation Analysis in Deep NLP Models" that we are presenting at NAACL 2021.

Domain Adaptation

Analyzing Individual Neurons in Pre-trained Language Models

1 code implementation EMNLP 2020 Nadir Durrani, Hassan Sajjad, Fahim Dalvi, Yonatan Belinkov

We found small subsets of neurons to predict linguistic tasks, with lower level tasks (such as morphology) localized in fewer neurons, compared to higher level task of predicting syntax.

Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms

1 code implementation15 Jul 2020 Firoj Alam, Fahim Dalvi, Shaden Shaar, Nadir Durrani, Hamdy Mubarak, Alex Nikolov, Giovanni Da San Martino, Ahmed Abdelali, Hassan Sajjad, Kareem Darwish, Preslav Nakov

With the outbreak of the COVID-19 pandemic, people turned to social media to read and to share timely information including statistics, warnings, advice, and inspirational stories.

Misinformation

FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN

no code implementations WS 2020 Ebrahim Ansari, Amittai Axelrod, Nguyen Bach, Ond{\v{r}}ej Bojar, Roldano Cattoni, Fahim Dalvi, Nadir Durrani, Marcello Federico, Christian Federmann, Jiatao Gu, Fei Huang, Kevin Knight, Xutai Ma, Ajay Nagesh, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Xing Shi, Sebastian St{\"u}ker, Marco Turchi, Alex Waibel, er, Changhan Wang

The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation.

Translation

Similarity Analysis of Contextual Word Representation Models

1 code implementation ACL 2020 John M. Wu, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, James Glass

We use existing and novel similarity measures that aim to gauge the level of localization of information in the deep models, and facilitate the investigation of which design factors affect model similarity, without requiring any external linguistic annotation.

Analyzing Redundancy in Pretrained Transformer Models

1 code implementation EMNLP 2020 Fahim Dalvi, Hassan Sajjad, Nadir Durrani, Yonatan Belinkov

Transformer-based deep NLP models are trained using hundreds of millions of parameters, limiting their applicability in computationally constrained environments.

Transfer Learning

On the Effect of Dropping Layers of Pre-trained Transformer Models

4 code implementations8 Apr 2020 Hassan Sajjad, Fahim Dalvi, Nadir Durrani, Preslav Nakov

Transformer-based NLP models are trained using hundreds of millions or even billions of parameters, limiting their applicability in computationally constrained environments.

Knowledge Distillation Sentence +1

One Size Does Not Fit All: Comparing NMT Representations of Different Granularities

no code implementations NAACL 2019 Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Yonatan Belinkov, Preslav Nakov

Recent work has shown that contextualized word representations derived from neural machine translation are a viable alternative to such from simple word predictions tasks.

Machine Translation NMT +1

NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks

2 code implementations21 Dec 2018 Fahim Dalvi, Avery Nortonsmith, D. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, James Glass

We present a toolkit to facilitate the interpretation and understanding of neural network models.

What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models

1 code implementation21 Dec 2018 Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Anthony Bau, James Glass

We further present a comprehensive analysis of neurons with the aim to address the following questions: i) how localized or distributed are different linguistic properties in the models?

Language Modelling Machine Translation +1

Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translation

no code implementations NAACL 2018 Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Stephan Vogel

We address the problem of simultaneous translation by modifying the Neural MT decoder to operate with dynamically built encoder and attention.

Machine Translation Translation

Continuous Space Reordering Models for Phrase-based MT

no code implementations IWSLT 2017 Nadir Durrani, Fahim Dalvi

We also observed improvements compared to the systems that used POS tags and word clusters to train these models.

POS

Understanding and Improving Morphological Learning in the Neural Machine Translation Decoder

no code implementations IJCNLP 2017 Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Stephan Vogel

End-to-end training makes the neural machine translation (NMT) architecture simpler, yet elegant compared to traditional statistical machine translation (SMT).

Machine Translation Multi-Task Learning +2

Neural Machine Translation Training in a Multi-Domain Scenario

no code implementations IWSLT 2017 Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Yonatan Belinkov, Stephan Vogel

Model stacking works best when training begins with the furthest out-of-domain data and the model is incrementally fine-tuned with the next furthest domain and so on.

Machine Translation Translation

Machine Translation Approaches and Survey for Indian Languages

no code implementations16 Jan 2017 Nadeem Jadoon Khan, Waqas Anwar, Nadir Durrani

We report the performance of baseline systems translating from Indian languages (Bengali, Guajarati, Hindi, Malayalam, Punjabi, Tamil, Telugu and Urdu) into English with average 10% accurate results for all the language pairs.

Machine Translation Translation

QCRI Machine Translation Systems for IWSLT 16

no code implementations14 Jan 2017 Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Stephan Vogel

This paper describes QCRI's machine translation systems for the IWSLT 2016 evaluation campaign.

Domain Adaptation Language Modelling +3

Cannot find the paper you are looking for? You can Submit a new open access paper.