Search Results for author: Ramesh Nallapati

Found 39 papers, 17 papers with code

H2KGAT: Hierarchical Hyperbolic Knowledge Graph Attention Network

no code implementations EMNLP 2020 Shen Wang, Xiaokai Wei, Cicero Nogueira dos santos, Zhiguo Wang, Ramesh Nallapati, Andrew Arnold, Bing Xiang, Philip S. Yu

Existing knowledge graph embedding approaches concentrate on modeling symmetry/asymmetry, inversion, and composition typed relations but overlook the hierarchical nature of relations.

Graph Attention Knowledge Graph Embedding +2

ReCode: Robustness Evaluation of Code Generation Models

2 code implementations20 Dec 2022 Shiqi Wang, Zheng Li, Haifeng Qian, Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, Bing Xiang

Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.

Code Generation

CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context

no code implementations20 Dec 2022 Yangruibo Ding, Zijian Wang, Wasi Uddin Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, Bing Xiang

While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i. e., in-file context, but ignore the rich semantics in other files within the same project, i. e., cross-file context, a critical source of information that is especially useful in modern modular software development.

Code Completion

Multi-lingual Evaluation of Code Generation Models

1 code implementation26 Oct 2022 Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.

Code Completion Code Translation +1

Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding

no code implementations28 Sep 2022 Jun Wang, Patrick Ng, Alexander Hanbo Li, Jiarong Jiang, Zhiguo Wang, Ramesh Nallapati, Bing Xiang, Sudipta Sengupta

When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.

NER Semantic Parsing +1

Efficient Few-Shot Fine-Tuning for Opinion Summarization

1 code implementation Findings (NAACL) 2022 Arthur Bražinskas, Ramesh Nallapati, Mohit Bansal, Markus Dreyer

In the same vein, we pre-train the adapters in a query-based manner on customer reviews and then fine-tune them on annotated datasets.

Abstractive Text Summarization

DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization

2 code implementations ACL 2022 Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth

Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.

Knowledge Distillation Model Compression +2

Pairwise Supervised Contrastive Learning of Sentence Representations

1 code implementation EMNLP 2021 Dejiao Zhang, Shang-Wen Li, Wei Xiao, Henghui Zhu, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang

Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese loss.

Contrastive Learning Natural Language Inference +3

Transductive Learning for Abstractive News Summarization

no code implementations17 Apr 2021 Arthur Bražinskas, Mengwen Liu, Ramesh Nallapati, Sujith Ravi, Markus Dreyer

This applies to scenarios such as a news publisher training a summarizer on dated news and summarizing incoming recent news.

Abstractive Text Summarization News Summarization +1

Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering

no code implementations EACL 2021 Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang

Experiments show that: (1) Our IR-based retrieval method is able to collect high-quality candidates efficiently, thus enables our method adapt to large-scale KBs easily; (2) the BERT model improves the accuracy across all three sub-tasks; and (3) benefiting from multi-task learning, the unified model obtains further improvements with only 1/3 of the original parameters.

Entity Linking Information Retrieval +4

Beyond [CLS] through Ranking by Generation

no code implementations EMNLP 2020 Cicero Nogueira dos santos, Xiaofei Ma, Ramesh Nallapati, Zhiheng Huang, Bing Xiang

Generative models for Information Retrieval, where ranking of documents is viewed as the task of generating a query from a document's language model, were very successful in various IR tasks in the past.

Answer Selection Information Retrieval +4

Embedding-based Zero-shot Retrieval through Query Generation

1 code implementation22 Sep 2020 Davis Liang, Peng Xu, Siamak Shakeri, Cicero Nogueira dos Santos, Ramesh Nallapati, Zhiheng Huang, Bing Xiang

In some cases, our model trained on synthetic data can even outperform the same model trained on real data

Passage Retrieval Retrieval

Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering

1 code implementation ACL 2020 Alexander R. Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

Training a QA model on this data gives a relative improvement over a previous unsupervised model in F1 score on the SQuAD dataset by about 14%, and 20% when the answer is a named entity, achieving state-of-the-art performance on SQuAD for unsupervised QA.

Language Modelling Question Answering +2

Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer

1 code implementation25 Nov 2019 Henghui Zhu, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

In this work, we define the problem of conversation structure modeling as identifying the parent utterance(s) to which each utterance in the conversation responds to.

Domain Adaptation with BERT-based Domain Classification and Data Selection

no code implementations WS 2019 Xiaofei Ma, Peng Xu, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

The performance of deep neural models can deteriorate substantially when there is a domain shift between training and test data.

Classification Domain Adaptation +1

Universal Text Representation from BERT: An Empirical Study

no code implementations17 Oct 2019 Xiaofei Ma, Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang

We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transferable they are across different tasks.

Learning-To-Rank Natural Language Inference +3

Multi Sense Embeddings from Topic Models

no code implementations WS 2019 Shobhit Jain, Sravan Babu Bodapati, Ramesh Nallapati, Anima Anandkumar

Distributed word embeddings have yielded state-of-the-art performance in many NLP tasks, mainly due to their success in capturing useful semantic information.

Topic Models Word Embeddings +1

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

no code implementations IJCNLP 2019 Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, Bing Xiang

To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages.

Open-Domain Question Answering

Topic Modeling with Wasserstein Autoencoders

1 code implementation ACL 2019 Feng Nan, Ran Ding, Ramesh Nallapati, Bing Xiang

To measure the diversity of the produced topics, we propose a simple topic uniqueness metric.

Topic Models

Passage Ranking with Weak Supervision

no code implementations ICLR Workshop LLD 2019 Peng Xu, Xiaofei Ma, Ramesh Nallapati, Bing Xiang

In this paper, we propose a \textit{weak supervision} framework for neural ranking tasks based on the data programming paradigm \citep{Ratner2016}, which enables us to leverage multiple weak supervision signals from different sources.

Passage Ranking


no code implementations ICLR Workshop LLD 2019 Ian Gemp, Ramesh Nallapati, Ran Ding, Feng Nan, Bing Xiang

We extend NTMs to the weakly semi-supervised setting by using informative priors in the training objective.

Topic Models

Coherence-Aware Neural Topic Modeling

2 code implementations EMNLP 2018 Ran Ding, Ramesh Nallapati, Bing Xiang

Topic models are evaluated based on their ability to describe documents well (i. e. low perplexity) and to produce topics that carry coherent semantic meaning.

Topic Models Variational Inference

SenGen: Sentence Generating Neural Variational Topic Model

no code implementations1 Aug 2017 Ramesh Nallapati, Igor Melnyk, Abhishek Kumar, Bo-Wen Zhou

We present a new topic model that generates documents by sampling a topic for one whole sentence at a time, and generating the words in the sentence using an RNN decoder that is conditioned on the topic of the sentence.

Classify or Select: Neural Architectures for Extractive Document Summarization

no code implementations14 Nov 2016 Ramesh Nallapati, Bo-Wen Zhou, Mingbo Ma

The Selector architecture, on the other hand, is free to pick one sentence at a time in any arbitrary order to piece together the summary.

Document Summarization Extractive Document Summarization +2

SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents

7 code implementations14 Nov 2016 Ramesh Nallapati, FeiFei Zhai, Bo-Wen Zhou

We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to state-of-the-art.

Document Summarization Extractive Summarization

Pointing the Unknown Words

no code implementations ACL 2016 Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bo-Wen Zhou, Yoshua Bengio

At each time-step, the decision of which softmax layer to use choose adaptively made by an MLP which is conditioned on the context.~We motivate our work from a psychological evidence that humans naturally have a tendency to point towards objects in the context or the environment when the name of an object is not known.~We observe improvements on two tasks, neural machine translation on the Europarl English to French parallel corpora and text summarization on the Gigaword dataset using our proposed model.

Machine Translation Text Summarization +1

Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond

4 code implementations CONLL 2016 Ramesh Nallapati, Bo-Wen Zhou, Cicero Nogueira dos santos, Caglar Gulcehre, Bing Xiang

In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora.

Abstractive Text Summarization Sentence Summarization +1

Cannot find the paper you are looking for? You can Submit a new open access paper.