Search Results for author:
Found 85 papers, 34 papers with code
H2KGAT: Hierarchical Hyperbolic Knowledge Graph Attention Network
Existing knowledge graph embedding approaches concentrate on modeling symmetry/asymmetry, inversion, and composition typed relations but overlook the hierarchical nature of relations.
Token Alignment via Character Matching for Subword Completion
Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens.
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios.
Code Representation Learning At Scale
Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i. e., code generation.
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning
We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data.
Lightweight reranking for language model generations
We show strong improvements for selecting the best k generations for code generation tasks as well as robust improvements for the best generation for the tasks of autoformalization, summarization, and translation.
Exploring Continual Learning for Code Generation Models
Large-scale code generation models such as Codex and CodeT5 have achieved impressive performance.
A Static Evaluation of Code Completion by Large Language Models
Large language models trained on code have shown great potential to increase productivity of software developers.
Efficient Shapley Values Estimation by Amortization for Text Classification
In practice, Shapley Values are often estimated with a small number of stochastic model evaluations.
Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge
The open-ended Visual Question Answering (VQA) task requires AI models to jointly reason over visual and natural language inputs using world knowledge.
Benchmarking Diverse-Modal Entity Linking with Generative Models
Entities can be expressed in diverse formats, such as texts, images, or column names and cell values in tables.
UNITE: A Unified Benchmark for Text-to-SQL Evaluation
A practical text-to-SQL system should generalize well on a wide variety of natural language questions, unseen database schemas, and novel SQL query structures.
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Such large models incur significant resource usage (in terms of memory, latency, and dollars) as well as carbon footprint.
STREET: A Multi-Task Structured Reasoning and Explanation Benchmark
We introduce STREET, a unified multi-task and multi-domain natural language reasoning and explanation benchmark.
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Neural text-to-SQL models have achieved remarkable performance in translating natural language questions into SQL queries.
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context
While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i. e., in-file context, but ignore the rich semantics in other files within the same project, i. e., cross-file context, a critical source of information that is especially useful in modern modular software development.
ReCode: Robustness Evaluation of Code Generation Models
Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.
Importance of Synthesizing High-quality Data for Text-to-SQL Parsing
In this paper, we first examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data.
Multi-lingual Evaluation of Code Generation Models
Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.
ContraCLM: Contrastive Learning For Causal Language Model
Specifically, we attain $44\%$ relative improvement on the Semantic Textual Similarity tasks and $34\%$ on Code-to-Code Search tasks.
DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases
Question answering over knowledge bases (KBs) aims to answer natural language questions with factual information such as entities and relations in KBs.
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding
When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.
REKnow: Enhanced Knowledge for Joint Entity and Relation Extraction
Our generative model is a unified framework to sequentially generate relational triplets under various relation extraction settings and explicitly utilizes relevant knowledge from Knowledge Graph (KG) to resolve ambiguities.
Learning Dialogue Representations from Consecutive Utterances
In this paper, we introduce Dialogue Sentence Embedding (DSE), a self-supervised contrastive learning method that learns effective dialogue representations suitable for a wide range of dialogue tasks.
DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization
Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.
Contrastive Document Representation Learning with Graph Attention Networks
In this work, we propose to use a graph attention network on top of the available pretrained Transformers model to learn document embeddings.
Attention-guided Generative Models for Extractive Question Answering
We propose a simple strategy to obtain an extractive answer span from the generative model by leveraging the decoder cross-attention patterns.
Multiplicative Position-aware Transformer Models for Language Understanding
Transformer models, which leverage architectural improvements like self-attention, perform remarkably well on Natural Language Processing (NLP) tasks.
Pairwise Supervised Contrastive Learning of Sentence Representations
Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese loss.
Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering
However, a large amount of world's knowledge is stored in structured databases, and need to be accessed using query languages such as SQL.
Joint Text and Label Generation for Spoken Language Understanding
In this work, we propose to leverage the prior information embedded in pretrained language models (LM) to improve generalization for intent classification and slot labeling tasks with limited training data.
Improving Factual Consistency of Abstractive Summarization via Question Answering
A commonly observed problem with the state-of-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents.
Generative Context Pair Selection for Multi-hop Question Answering
Compositional reasoning tasks like multi-hop question answering, require making latent decisions to get the final answer, given a question.
Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering
Experiments show that: (1) Our IR-based retrieval method is able to collect high-quality candidates efficiently, thus enables our method adapt to large-scale KBs easily; (2) the BERT model improves the accuracy across all three sub-tasks; and (3) benefiting from multi-task learning, the unified model obtains further improvements with only 1/3 of the original parameters.
Supporting Clustering with Contrastive Learning
Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space.
Ranked #1 on
Short Text Clustering
on AG News
Entity-level Factual Consistency of Abstractive Text Summarization
A key challenge for abstractive summarization is ensuring factual consistency of the generated summary with respect to the original document.
Structured Prediction as Translation between Augmented Natural Languages
We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks including joint entity and relation extraction, nested named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, and dialogue state tracking.
Ranked #3 on
Relation Classification
on TACRED
Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training
Most recently, there has been significant interest in learning contextual representations for various NLP tasks, by leveraging large scale text corpora to train large neural language models with self-supervised learning objectives, such as Masked Language Model (MLM).
Ranked #7 on
Semantic Parsing
on spider
Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction
When multiple plausible answers are found, the system should rewrite the question for each answer to resolve the ambiguity.
Margin-aware Unsupervised Domain Adaptation for Cross-lingual Text Labeling
Unsupervised domain adaptation addresses the problem of leveraging labeled data in a source domain to learn a well-performing model in a target domain where labels are unavailable.
Cross-Lingual Document Classification
Document Classification
+2
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems
Our model comprises a single transformer-based encoder-decoder network that is trained end-to-end to generate both answers and questions.
Beyond [CLS] through Ranking by Generation
Generative models for Information Retrieval, where ranking of documents is viewed as the task of generating a query from a document's language model, were very successful in various IR tasks in the past.
Improve Transformer Models with Better Relative Position Embeddings
In this paper, we first review absolute position embeddings and existing methods for relative position embeddings.
Embedding-based Zero-shot Retrieval through Query Generation
In some cases, our model trained on synthetic data can even outperform the same model trained on real data
Augmented Natural Language for Generative Sequence Labeling
We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot ($75. 0\% \rightarrow 90. 9\%$) and 1-shot ($70. 4\% \rightarrow 81. 0\%$) state-of-the-art results.
AWS CORD-19 Search: A Neural Search Engine for COVID-19 Literature
Coronavirus disease (COVID-19) has been declared as a pandemic by WHO with thousands of cases being reported each day.
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
Training a QA model on this data gives a relative improvement over a previous unsupervised model in F1 score on the SQuAD dataset by about 14%, and 20% when the answer is a named entity, achieving state-of-the-art performance on SQuAD for unsupervised QA.
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
Prior to the transformer era, bidirectional Long Short-Term Memory (BLSTM) has been the dominant modeling architecture for neural machine translation and question answering.
Ranked #1 on
Text Classification
on GLUE MRPC
Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer
In this work, we define the problem of conversation structure modeling as identifying the parent utterance(s) to which each utterance in the conversation responds to.
Domain Adaptation with BERT-based Domain Classification and Data Selection
The performance of deep neural models can deteriorate substantially when there is a domain shift between training and test data.
Universal Text Representation from BERT: An Empirical Study
We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transferable they are across different tasks.
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages.
Ranked #3 on
Open-Domain Question Answering
on SearchQA
Topic Modeling with Wasserstein Autoencoders
To measure the diversity of the produced topics, we propose a simple topic uniqueness metric.
Passage Ranking with Weak Supervision
In this paper, we propose a \textit{weak supervision} framework for neural ranking tasks based on the data programming paradigm \citep{Ratner2016}, which enables us to leverage multiple weak supervision signals from different sources.
WeNet: Weighted Networks for Recurrent Network Architecture Search
In this paper, we propose a novel way of architecture search by means of weighted networks (WeNet), which consist of a number of networks, with each assigned a weight.
OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations
The key contribution of our work is our proposal to explicitly constrain the latent space to exclusively represent the given class.
Ranked #6 on
Anomaly Detection
on Hyper-Kvasir Dataset
WEAKLY SEMI-SUPERVISED NEURAL TOPIC MODELS
We extend NTMs to the weakly semi-supervised setting by using informative priors in the training objective.
Coherence-Aware Neural Topic Modeling
Topic models are evaluated based on their ability to describe documents well (i. e. low perplexity) and to produce topics that carry coherent semantic meaning.
Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering
Coreference resolution aims to identify in a text all mentions that refer to the same real-world entity.
Group Sparse CNNs for Question Classification with Answer Sets
Question classification is an important task with wide applications.
Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks
In order to utilize the potential benefits from their correlations, we propose a jointly trained model for learning the two tasks simultaneously via Long Short-Term Memory (LSTM) networks.
Improved Neural Relation Detection for Knowledge Base Question Answering
Relation detection is a core component for many NLP applications including Knowledge Base Question Answering (KBQA).
A Structured Self-attentive Sentence Embedding
This paper proposes a new model for extracting an interpretable sentence embedding by introducing self-attention.
Neural Models for Sequence Chunking
Many natural language understanding (NLU) tasks, such as shallow parsing (i. e., text chunking) and semantic slot filling, require the assignment of representative labels to the meaningful chunks in a sentence.
GaDei: On Scale-up Training As A Service For Deep Learning
By evaluating the NLC workloads, we show that only the conservative hyper-parameter setup (e. g., small mini-batch size and small learning rate) can guarantee acceptable model accuracy for a wide range of customers.
End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension
This paper proposes dynamic chunk reader (DCR), an end-to-end neural reading comprehension (RC) model that is able to extract and rank a set of answer candidates from a given document to answer questions.
Ranked #49 on
Question Answering
on SQuAD1.1 dev
Simple Question Answering by Attentive Convolutional Neural Network
In fact selection, we match the subject entity in a fact candidate with the entity mention in the question by a character-level convolutional neural network (char-CNN), and match the predicate in that fact with the question by a word-level CNN (word-CNN).
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora.
Ranked #10 on
Text Summarization
on DUC 2004 Task 1
Attentive Pooling Networks
In this work, we propose Attentive Pooling (AP), a two-way attention mechanism for discriminative model training.
Ranked #2 on
Question Answering
on SemEvalCQA
Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling
Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling.
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
(ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart.
Good, Better, Best: Choosing Word Embedding Context
We propose two methods of learning vector representations of words and phrases that each combine sentence context with structural features extracted from dependency trees.
LSTM-based Deep Learning Models for Non-factoid Answer Selection
One direction is to define a more composite representation for questions and answers by combining convolutional neural network with the basic framework.
Distributed Deep Learning for Question Answering
This paper is an empirical study of the distributed deep learning for question answering subtasks: answer selection and question classification.
Empirical Study on Deep Learning Models for Question Answering
In this paper we explore deep learning models with memory component or attention mechanism for question answering task.
Applying Deep Learning to Answer Selection: A Study and An Open Task
We apply a general deep learning framework to address the non-factoid question answering task.
Dependency-based Convolutional Neural Networks for Sentence Embedding
In sentence modeling and classification, convolutional neural network approaches have recently achieved state-of-the-art results, but all such efforts process word vectors sequentially and neglect long-distance dependencies.
Classifying Relations by Ranking with Convolutional Neural Networks
Relation classification is an important semantic processing task for which state-ofthe-art systems still rely on costly handcrafted features.
Ranked #27 on
Relation Extraction
on SemEval-2010 Task-8
