Search Results for author:
Found 15 papers, 7 papers with code
KFCNet: Knowledge Filtering and Contrastive Learning for Generative Commonsense Reasoning
Pre-trained language models have led to substantial gains over a broad range of natural language processing (NLP) tasks, but have been shown to have limitations for natural language generation tasks with high-quality requirements on the output, such as commonsense generation and ad keyword generation.
A Self-Paced Mixed Distillation Method for Non-Autoregressive Generation
To further illustrate the commercial value of our approach, we conduct experiments on three generation tasks in real-world advertisements applications.
Taming Sparsely Activated Transformer with Stochastic Experts
While most on-going research focuses on improving SAMs models by exploring methods of routing inputs to experts, our analysis reveals that such research might not lead to the solution we expect, i. e., the commonly-used routing methods based on gating mechanisms do not work better than randomly routing inputs to experts.
KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
Pre-trained language models have led to substantial gains over a broad range of natural language processing (NLP) tasks, but have been shown to have limitations for natural language generation tasks with high-quality requirements on the output, such as commonsense generation and ad keyword generation.
GalaXC: Graph Neural Networks with Labelwise Attention for Extreme Classification
An efficient end-to-end implementation of GalaXC is presented that could be trained on a dataset with 50M labels and 97M training documents in less than 100 hours on 4×V100 GPUs.
Mask Attention Networks: Rethinking and Strengthen Transformer
We therefore introduce a new layer named dynamic mask attention network (DMAN) with a learnable mask matrix which is able to model localness adaptively.
Ranked #9 on
Machine Translation
on WMT2014 English-German
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation.
An Enhanced Knowledge Injection Model for Commonsense Generation
Commonsense generation aims at generating plausible everyday scenario description based on a set of provided concepts.
GLGE: A New General Language Generation Evaluation Benchmark
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP).
Natural Language Processing
Natural Language Understanding
+2
ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
We build a dataset from a real-word sponsored search engine and carry out experiments to analyze different generative retrieval models.
HittER: Hierarchical Transformers for Knowledge Graph Embeddings
Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block.
TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
Experimental results showed that the inference time was significantly reduced and was firstly controlled around 20ms on CPUs while at the same time the performance gain from fine-tuned BERT-Base model was mostly retained.
MagicVO: End-to-End Monocular Visual Odometry through Deep Bi-directional Recurrent Convolutional Neural Network
This paper proposes a new framework to solve the problem of monocular visual odometry, called MagicVO .
Recurrent Binary Embedding for GPU-Enabled Exhaustive Retrieval from Billion-Scale Semantic Vectors
Building on top of the powerful concept of semantic learning, this paper proposes a Recurrent Binary Embedding (RBE) model that learns compact representations for real-time retrieval.
Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features
Manually crafted combinatorial features have been the “secret sauce” behind many successful models.
