Search Results for author:
Found 43 papers, 19 papers with code
H2KGAT: Hierarchical Hyperbolic Knowledge Graph Attention Network
Existing knowledge graph embedding approaches concentrate on modeling symmetry/asymmetry, inversion, and composition typed relations but overlook the hierarchical nature of relations.
BASS: Batched Attention-optimized Speculative Sampling
Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models.
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios.
Token Alignment via Character Matching for Subword Completion
Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens.
Code Representation Learning At Scale
Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i. e., code generation.
Exploring Continual Learning for Code Generation Models
Large-scale code generation models such as Codex and CodeT5 have achieved impressive performance.
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context
While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i. e., in-file context, but ignore the rich semantics in other files within the same project, i. e., cross-file context, a critical source of information that is especially useful in modern modular software development.
ReCode: Robustness Evaluation of Code Generation Models
Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.
Multi-lingual Evaluation of Code Generation Models
Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.
ContraCLM: Contrastive Learning For Causal Language Model
Specifically, we attain $44\%$ relative improvement on the Semantic Textual Similarity tasks and $34\%$ on Code-to-Code Search tasks.
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding
When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.
Efficient Few-Shot Fine-Tuning for Opinion Summarization
In the same vein, we pre-train the adapters in a query-based manner on customer reviews and then fine-tune them on annotated datasets.
DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization
Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.
Pairwise Supervised Contrastive Learning of Sentence Representations
Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese loss.
Improving Factual Consistency of Abstractive Summarization via Question Answering
A commonly observed problem with the state-of-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents.
Transductive Learning for Abstractive News Summarization
This applies to scenarios such as a news publisher training a summarizer on dated news and summarizing incoming recent news.
Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering
Experiments show that: (1) Our IR-based retrieval method is able to collect high-quality candidates efficiently, thus enables our method adapt to large-scale KBs easily; (2) the BERT model improves the accuracy across all three sub-tasks; and (3) benefiting from multi-task learning, the unified model obtains further improvements with only 1/3 of the original parameters.
Supporting Clustering with Contrastive Learning
Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space.
Ranked #1 on
Short Text Clustering
on AG News
Entity-level Factual Consistency of Abstractive Text Summarization
A key challenge for abstractive summarization is ensuring factual consistency of the generated summary with respect to the original document.
Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction
When multiple plausible answers are found, the system should rewrite the question for each answer to resolve the ambiguity.
Margin-aware Unsupervised Domain Adaptation for Cross-lingual Text Labeling
Unsupervised domain adaptation addresses the problem of leveraging labeled data in a source domain to learn a well-performing model in a target domain where labels are unavailable.
Cross-Lingual Document Classification
Document Classification
+2
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems
Our model comprises a single transformer-based encoder-decoder network that is trained end-to-end to generate both answers and questions.
Beyond [CLS] through Ranking by Generation
Generative models for Information Retrieval, where ranking of documents is viewed as the task of generating a query from a document's language model, were very successful in various IR tasks in the past.
Embedding-based Zero-shot Retrieval through Query Generation
In some cases, our model trained on synthetic data can even outperform the same model trained on real data
AWS CORD-19 Search: A Neural Search Engine for COVID-19 Literature
Coronavirus disease (COVID-19) has been declared as a pandemic by WHO with thousands of cases being reported each day.
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
Training a QA model on this data gives a relative improvement over a previous unsupervised model in F1 score on the SQuAD dataset by about 14%, and 20% when the answer is a named entity, achieving state-of-the-art performance on SQuAD for unsupervised QA.
Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer
In this work, we define the problem of conversation structure modeling as identifying the parent utterance(s) to which each utterance in the conversation responds to.
Domain Adaptation with BERT-based Domain Classification and Data Selection
The performance of deep neural models can deteriorate substantially when there is a domain shift between training and test data.
Universal Text Representation from BERT: An Empirical Study
We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transferable they are across different tasks.
Multi Sense Embeddings from Topic Models
Distributed word embeddings have yielded state-of-the-art performance in many NLP tasks, mainly due to their success in capturing useful semantic information.
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages.
Ranked #3 on
Open-Domain Question Answering
on SearchQA
Topic Modeling with Wasserstein Autoencoders
To measure the diversity of the produced topics, we propose a simple topic uniqueness metric.
Passage Ranking with Weak Supervision
In this paper, we propose a \textit{weak supervision} framework for neural ranking tasks based on the data programming paradigm \citep{Ratner2016}, which enables us to leverage multiple weak supervision signals from different sources.
OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations
The key contribution of our work is our proposal to explicitly constrain the latent space to exclusively represent the given class.
Ranked #6 on
Anomaly Detection
on Hyper-Kvasir Dataset
WEAKLY SEMI-SUPERVISED NEURAL TOPIC MODELS
We extend NTMs to the weakly semi-supervised setting by using informative priors in the training objective.
Coherence-Aware Neural Topic Modeling
Topic models are evaluated based on their ability to describe documents well (i. e. low perplexity) and to produce topics that carry coherent semantic meaning.
SenGen: Sentence Generating Neural Variational Topic Model
We present a new topic model that generates documents by sampling a topic for one whole sentence at a time, and generating the words in the sentence using an RNN decoder that is conditioned on the topic of the sentence.
SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to state-of-the-art.
Ranked #8 on
Text Summarization
on CNN / Daily Mail (Anonymized)
Classify or Select: Neural Architectures for Extractive Document Summarization
The Selector architecture, on the other hand, is free to pick one sentence at a time in any arbitrary order to piece together the summary.
Pointing the Unknown Words
At each time-step, the decision of which softmax layer to use choose adaptively made by an MLP which is conditioned on the context.~We motivate our work from a psychological evidence that humans naturally have a tendency to point towards objects in the context or the environment when the name of an object is not known.~We observe improvements on two tasks, neural machine translation on the Europarl English to French parallel corpora and text summarization on the Gigaword dataset using our proposed model.
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora.
Ranked #10 on
Text Summarization
on DUC 2004 Task 1
