Search Results for author: Kalpesh Krishna

Found 25 papers, 19 papers with code

GEE! Grammar Error Explanation with Large Language Models

1 code implementation16 Nov 2023 Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Kevin Gimpel, Mohit Iyyer

To address this gap, we propose the task of grammar error explanation, where a system needs to provide one-sentence explanations for each grammatical error in a pair of erroneous and corrected sentences.

Grammatical Error Correction

FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation

1 code implementation23 May 2023 Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi

Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly.

Language Modelling Retrieval +1

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

1 code implementation NeurIPS 2023 Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, Mohit Iyyer

To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider.

Language Modelling Outlier Detection +3

Stealing the Decoding Algorithms of Language Models

1 code implementation8 Mar 2023 Ali Naseh, Kalpesh Krishna, Mohit Iyyer, Amir Houmansadr

A key component of generating text from modern language models (LM) is the selection and tuning of decoding algorithms.

Text Generation

LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization

1 code implementation30 Jan 2023 Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, Kyle Lo

Motivated by our survey, we present LongEval, a set of guidelines for human evaluation of faithfulness in long-form summaries that addresses the following challenges: (1) How can we achieve high inter-annotator agreement on faithfulness scores?

Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature

1 code implementation25 Oct 2022 Katherine Thai, Marzena Karpinska, Kalpesh Krishna, Bill Ray, Moira Inghilleri, John Wieting, Mohit Iyyer

Using Par3, we discover that expert literary translators prefer reference human translations over machine-translated paragraphs at a rate of 84%, while state-of-the-art automatic MT metrics do not correlate with those preferences.

Machine Translation Translation

SLING: Sino Linguistic Evaluation of Large Language Models

1 code implementation21 Oct 2022 Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Mohit Iyyer

To understand what kinds of linguistic knowledge are encoded by pretrained Chinese language models (LMs), we introduce the benchmark of Sino LINGuistics (SLING), which consists of 38K minimal sentence pairs in Mandarin Chinese grouped into 9 high-level linguistic phenomena.

RankGen: Improving Text Generation with Large Ranking Models

1 code implementation19 May 2022 Kalpesh Krishna, Yapei Chang, John Wieting, Mohit Iyyer

Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts.

Contrastive Learning Language Modelling +2

RELIC: Retrieving Evidence for Literary Claims

1 code implementation ACL 2022 Katherine Thai, Yapei Chang, Kalpesh Krishna, Mohit Iyyer

Humanities scholars commonly provide evidence for claims that they make about a work of literature (e. g., a novel) in the form of quotations from the work.

Information Retrieval Retrieval +2

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation

2 code implementations6 Dec 2021 Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahendiran, Simon Mille, Ashish Shrivastava, Samson Tan, Tongshuang Wu, Jascha Sohl-Dickstein, Jinho D. Choi, Eduard Hovy, Ondrej Dusek, Sebastian Ruder, Sajant Anand, Nagender Aneja, Rabin Banjade, Lisa Barthe, Hanna Behnke, Ian Berlot-Attwell, Connor Boyle, Caroline Brun, Marco Antonio Sobrevilla Cabezudo, Samuel Cahyawijaya, Emile Chapuis, Wanxiang Che, Mukund Choudhary, Christian Clauss, Pierre Colombo, Filip Cornell, Gautier Dagan, Mayukh Das, Tanay Dixit, Thomas Dopierre, Paul-Alexis Dray, Suchitra Dubey, Tatiana Ekeinhor, Marco Di Giovanni, Tanya Goyal, Rishabh Gupta, Louanes Hamla, Sang Han, Fabrice Harel-Canada, Antoine Honore, Ishan Jindal, Przemyslaw K. Joniak, Denis Kleyko, Venelin Kovatchev, Kalpesh Krishna, Ashutosh Kumar, Stefan Langer, Seungjae Ryan Lee, Corey James Levinson, Hualou Liang, Kaizhao Liang, Zhexiong Liu, Andrey Lukyanenko, Vukosi Marivate, Gerard de Melo, Simon Meoni, Maxime Meyer, Afnan Mir, Nafise Sadat Moosavi, Niklas Muennighoff, Timothy Sum Hon Mun, Kenton Murray, Marcin Namysl, Maria Obedkova, Priti Oli, Nivranshu Pasricha, Jan Pfister, Richard Plant, Vinay Prabhu, Vasile Pais, Libo Qin, Shahab Raji, Pawan Kumar Rajpoot, Vikas Raunak, Roy Rinberg, Nicolas Roberts, Juan Diego Rodriguez, Claude Roux, Vasconcellos P. H. S., Ananya B. Sai, Robin M. Schmidt, Thomas Scialom, Tshephisho Sefara, Saqib N. Shamsi, Xudong Shen, Haoyue Shi, Yiwen Shi, Anna Shvets, Nick Siegel, Damien Sileo, Jamie Simon, Chandan Singh, Roman Sitelew, Priyank Soni, Taylor Sorensen, William Soto, Aman Srivastava, KV Aditya Srivatsa, Tony Sun, Mukund Varma T, A Tabassum, Fiona Anting Tan, Ryan Teehan, Mo Tiwari, Marie Tolkiehn, Athena Wang, Zijian Wang, Gloria Wang, Zijie J. Wang, Fuxuan Wei, Bryan Wilie, Genta Indra Winata, Xinyi Wu, Witold Wydmański, Tianbao Xie, Usama Yaseen, Michael A. Yee, Jing Zhang, Yue Zhang

Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.

Data Augmentation

Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings

no code implementations ACL 2022 Kalpesh Krishna, Deepak Nathani, Xavier Garcia, Bidisha Samanta, Partha Talukdar

When compared to prior work, our model achieves 2-3x better performance in formality transfer and code-mixing addition across seven languages.

Style Transfer Text Anonymization +1

Do Long-Range Language Models Actually Use Long-Range Context?

no code implementations EMNLP 2021 Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, Mohit Iyyer

Language models are generally trained on short, truncated input sequences, which limits their ability to use discourse-level information present in long-range context to improve their predictions.

Hurdles to Progress in Long-form Question Answering

2 code implementations NAACL 2021 Kalpesh Krishna, Aurko Roy, Mohit Iyyer

The task of long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer.

Long Form Question Answering Open-Domain Dialog +2

An Analysis of Frame-skipping in Reinforcement Learning

no code implementations7 Feb 2021 Shivaram Kalyanakrishnan, Siddharth Aravindan, Vishwajeet Bagdawat, Varun Bhatt, Harshith Goka, Archit Gupta, Kalpesh Krishna, Vihari Piratla

In this paper, we investigate the role of the parameter $d$ in RL; $d$ is called the "frame-skip" parameter, since states in the Atari domain are images.

Decision Making reinforcement-learning +1

Reformulating Unsupervised Style Transfer as Paraphrase Generation

1 code implementation EMNLP 2020 Kalpesh Krishna, John Wieting, Mohit Iyyer

Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfer systems should be paraphrases of their inputs.

Paraphrase Generation Style Transfer

Thieves on Sesame Street! Model Extraction of BERT-based APIs

1 code implementation ICLR 2020 Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer

We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model.

Language Modelling Model extraction +3

Generating Question-Answer Hierarchies

2 code implementations ACL 2019 Kalpesh Krishna, Mohit Iyyer

The process of knowledge acquisition can be viewed as a question-answer game between a student and a teacher in which the student typically starts by asking broad, open-ended questions before drilling down into specifics (Hintikka, 1981; Hakkarainen and Sintonen, 2002).

Language Modelling Reading Comprehension +2

Syntactically Supervised Transformers for Faster Neural Machine Translation

1 code implementation ACL 2019 Nader Akoury, Kalpesh Krishna, Mohit Iyyer

Standard decoders for neural machine translation autoregressively generate a single target token per time step, which slows inference especially for long outputs.

Machine Translation Translation

Trick or TReAT: Thematic Reinforcement for Artistic Typography

1 code implementation19 Mar 2019 Purva Tendulkar, Kalpesh Krishna, Ramprasaath R. Selvaraju, Devi Parikh

An approach to make text visually appealing and memorable is semantic reinforcement - the use of visual cues alluding to the context or theme in which the word is being used to reinforce the message (e. g., Google Doodles).

Revisiting the Importance of Encoding Logic Rules in Sentiment Classification

1 code implementation EMNLP 2018 Kalpesh Krishna, Preethi Jyothi, Mohit Iyyer

We analyze the performance of different sentiment classification models on syntactically complex inputs like A-but-B sentences.

Classification General Classification +2

Hierarchical Multitask Learning for CTC-based Speech Recognition

no code implementations17 Jul 2018 Kalpesh Krishna, Shubham Toshniwal, Karen Livescu

Previous work has shown that neural encoder-decoder speech recognition can be improved with hierarchical multitask learning, where auxiliary tasks are added at intermediate layers of a deep encoder.

speech-recognition Speech Recognition +1

Cannot find the paper you are looking for? You can Submit a new open access paper.