Search Results for author: Ankur P. Parikh

Found 29 papers, 11 papers with code

Extrapolative Controlled Sequence Generation via Iterative Refinement

1 code implementation8 Mar 2023 Vishakh Padmakumar, Richard Yuanzhe Pang, He He, Ankur P. Parikh

We study the problem of extrapolative controlled generation, i. e., generating sequences with attribute values beyond the range seen in training.

Attribute Drug Discovery +1

Reward Gaming in Conditional Text Generation

no code implementations16 Nov 2022 Richard Yuanzhe Pang, Vishakh Padmakumar, Thibault Sellam, Ankur P. Parikh, He He

To align conditional text generation model outputs with desired behaviors, there has been an increasing focus on training the model using reinforcement learning (RL) with reward functions learned from human annotations.

Conditional Text Generation Reinforcement Learning (RL)

Amos: An Adam-style Optimizer with Adaptive Weight Decay towards Model-Oriented Scale

1 code implementation21 Oct 2022 Ran Tian, Ankur P. Parikh

We present Amos, a stochastic gradient-based optimizer designed for training deep neural networks.

SQuId: Measuring Speech Naturalness in Many Languages

no code implementations12 Oct 2022 Thibault Sellam, Ankur Bapna, Joshua Camp, Diana Mackinnon, Ankur P. Parikh, Jason Riesa

The main insight is that training one model on many locales consistently outperforms mono-locale baselines.


Simple Recurrence Improves Masked Language Models

no code implementations23 May 2022 Tao Lei, Ran Tian, Jasmijn Bastings, Ankur P. Parikh

In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into the Transformer.

Learning Compact Metrics for MT

1 code implementation EMNLP 2021 Amy Pu, Hyung Won Chung, Ankur P. Parikh, Sebastian Gehrmann, Thibault Sellam

Recent developments in machine translation and multilingual text generation have led researchers to adopt trained metrics such as COMET or BLEURT, which treat evaluation as a regression problem and use representations from multilingual pre-trained models such as XLM-RoBERTa or mBERT.

Cross-Lingual Transfer Language Modelling +4

Shatter: An Efficient Transformer Encoder with Single-Headed Self-Attention and Relative Sequence Partitioning

no code implementations30 Aug 2021 Ran Tian, Joshua Maynez, Ankur P. Parikh

The highly popular Transformer architecture, based on self-attention, is the foundation of large pretrained models such as BERT, that have become an enduring paradigm in NLP.

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

no code implementations NAACL 2021 Xavier Garcia, Noah Constant, Ankur P. Parikh, Orhan Firat

We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation.

Continual Learning Machine Translation +1

ToTTo: A Controlled Table-To-Text Generation Dataset

1 code implementation EMNLP 2020 Ankur P. Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, Dipanjan Das

We present ToTTo, an open-domain English table-to-text dataset with over 120, 000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description.

Conditional Text Generation Data-to-Text Generation +2

BLEURT: Learning Robust Metrics for Text Generation

3 code implementations ACL 2020 Thibault Sellam, Dipanjan Das, Ankur P. Parikh

We propose BLEURT, a learned evaluation metric based on BERT that can model human judgments with a few thousand possibly biased training examples.

Text Generation

A Multilingual View of Unsupervised Machine Translation

no code implementations Findings of the Association for Computational Linguistics 2020 Xavier Garcia, Pierre Foret, Thibault Sellam, Ankur P. Parikh

We present a probabilistic framework for multilingual neural machine translation that encompasses supervised and unsupervised setups, focusing on unsupervised translation.

Translation Unsupervised Machine Translation

Thieves on Sesame Street! Model Extraction of BERT-based APIs

1 code implementation ICLR 2020 Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer

We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model.

Language Modelling Model extraction +3

Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation

no code implementations19 Oct 2019 Ran Tian, Shashi Narayan, Thibault Sellam, Ankur P. Parikh

We address the issue of hallucination in data-to-text generation, i. e., reducing the generation of text that is unsupported by the source.

Data-to-Text Generation Decoder +1

Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index

1 code implementation ACL 2019 Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, Hannaneh Hajishirzi

Existing open-domain question answering (QA) models are not suitable for real-time usage because they need to process several long documents on-demand for every input query.

Open-Domain Question Answering

Consistency by Agreement in Zero-shot Neural Machine Translation

2 code implementations NAACL 2019 Maruan Al-Shedivat, Ankur P. Parikh

Generalization and reliability of multilingual translation often highly depend on the amount of available parallel data for each language pair of interest.

Machine Translation NMT +3

Hybrid Subspace Learning for High-Dimensional Data

no code implementations5 Aug 2018 Micol Marchetti-Bowick, Benjamin J. Lengerich, Ankur P. Parikh, Eric P. Xing

One way to achieve this goal is to perform subspace learning to estimate a small set of latent features that capture the majority of the variance in the original data.

Dimensionality Reduction Video Background Subtraction +1

Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension

1 code implementation EMNLP 2018 Minjoon Seo, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, Hannaneh Hajishirzi

We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder.

Question Answering Reading Comprehension +1

Multi-Mention Learning for Reading Comprehension with Neural Cascades

no code implementations ICLR 2018 Swabha Swayamdipta, Ankur P. Parikh, Tom Kwiatkowski

Reading comprehension is a challenging task, especially when executed across longer or across multiple evidence documents, where the answer is likely to reoccur.

Reading Comprehension TriviaQA

Infinite Mixed Membership Matrix Factorization

no code implementations15 Jan 2014 Avneesh Saluja, Mahdi Pakdaman, Dongzhen Piao, Ankur P. Parikh

Rating and recommendation systems have become a popular application area for applying a suite of machine learning techniques.

Recommendation Systems

Language Modeling with Power Low Rank Ensembles

no code implementations EMNLP 2014 Ankur P. Parikh, Avneesh Saluja, Chris Dyer, Eric P. Xing

We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context.

Language Modelling Machine Translation +1

Kernel Embeddings of Latent Tree Graphical Models

no code implementations NeurIPS 2011 Le Song, Eric P. Xing, Ankur P. Parikh

Latent tree graphical models are natural tools for expressing long range and hierarchical dependencies among many variables which are common in computer vision, bioinformatics and natural language processing problems.

Cannot find the paper you are looking for? You can Submit a new open access paper.