Search Results for author:
Found 39 papers, 24 papers with code
P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks
Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training.
Learning to Detect Noisy Labels Using Model-Based Features
Label noise is ubiquitous in various machine learning scenarios such as self-labeling with model predictions and erroneous data annotation.
A Universal Discriminator for Zero-Shot Generalization
In the finetuning setting, our approach also achieves new state-of-the-art results on a wide range of NLP tasks, with only 1/4 parameters of previous methods.
Zero-Label Prompt Selection
Natural language prompts have been shown to facilitate cross-task generalization for large language models.
Prompt-Based Metric Learning for Few-Shot NER
Few-shot named entity recognition (NER) targets generalizing to unseen labels and/or domains with few labeled examples.
GPS: Genetic Prompt Search for Efficient Few-shot Learning
Prompt-based techniques have demostrated great potential for improving the few-shot generalization of pretrained language models.
ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization
We propose a multitask pretraining approach ZeroPrompt for zero-shot generalization, focusing on task scaling and zero-shot prompting.
NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework
Pretrained language models have become the standard approach for many NLP tasks due to strong performance, but they are very expensive to train.
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training.
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding
The few-shot natural language understanding (NLU) task has attracted much recent attention.
FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning
Most previous methods for text data augmentation are limited to simple tasks and weak baselines.
Distribution Matching for Rationalization
The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks.
VeniBot: Towards Autonomous Venipuncture with Automatic Puncture Area and Angle Regression from NIR Images
For the automation, we focus on the positioning part and propose a Dual-In-Dual-Out network based on two-step learning and two-task learning, which can achieve fully automatic regression of the suitable puncture area and angle from near-infrared(NIR) images.
FastMoE: A Fast Mixture-of-Expert Training System
However, training trillion-scale MoE requires algorithm and system co-design for a well-tuned high performance distributed training system.
Controllable Generation from Pre-trained Language Models via Inverse Prompting
To tackle this challenge, we propose an innovative method, inverse prompting, to better control text generation.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
On a wide range of tasks across NLU, conditional and unconditional generation, GLM outperforms BERT, T5, and GPT given the same model sizes and data, and achieves the best performance from a single pretrained model with 1. 25x parameters of BERT Large , demonstrating its generalizability to different downstream tasks.
Ranked #2 on
Document Summarization
on CNN / Daily Mail
GPT Understands, Too
On the SuperGlue benchmark, GPTs achieve comparable and sometimes better performance to similar-sized BERTs in supervised learning.
Mixtape: Breaking the Softmax Bottleneck Efficiently
The softmax bottleneck has been shown to limit the expressiveness of neural language models.
XLNet: Generalized Autoregressive Pretraining for Language Understanding
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.
Ranked #3 on
Language Modelling
on One Billion Word
GLoMo: Unsupervised Learning of Transferable Relational Graphs
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden units), or embedding-free units such as image pixels.
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers.
Ranked #36 on
Question Answering
on HotpotQA
Neural Cross-Lingual Named Entity Recognition with Minimal Resources
To improve robustness to word order differences, we propose to use self-attention, which allows for a degree of flexibility with respect to word order.
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden unit), or embedding-free units such as image pixels.
Neural Models for Reasoning over Multiple Mentions using Coreference
Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text.
Ranked #6 on
Question Answering
on WikiHop
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent
Contrary to most natural language processing research, which makes use of static datasets, humans learn language interactively, grounded in an environment.
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck.
Ranked #10 on
Language Modelling
on Penn Treebank (Word Level)
Good Semi-supervised Learning that Requires a Bad GAN
Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time.
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks.
Ranked #10 on
Part-Of-Speech Tagging
on Penn Treebank
Linguistic Knowledge as Memory for Recurrent Neural Networks
We introduce a model that encodes such graphs as explicit memory in recurrent neural networks, and use it to model coreference relations in text.
Ranked #1 on
Question Answering
on CNN / Daily Mail
Differentiable Learning of Logical Rules for Knowledge Base Reasoning
We propose a framework, Neural Logic Programming, that combines the parameter and structure learning of first-order logical rules in an end-to-end differentiable model.
A Probabilistic Framework for Location Inference from Social Media
In this paper, we formalize the problem of inferring location from social media into a semi-supervised factor graph model (SSFGM).
Semi-Supervised QA with Generative Domain-Adaptive Nets
In this framework, we train a generative model to generate questions based on the unlabeled text, and combine model-generated questions with human-generated questions for training question answering models.
Words or Characters? Fine-grained Gating for Reading Comprehension
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension.
Ranked #50 on
Question Answering
on SQuAD1.1 dev
Gated-Attention Readers for Text Comprehension
In this paper we study the problem of answering cloze-style questions over documents.
Ranked #1 on
Question Answering
on Children's Book Test
Review Networks for Caption Generation
We propose a novel extension of the encoder-decoder framework, called a review network.
Revisiting Semi-Supervised Learning with Graph Embeddings
We present a semi-supervised learning framework based on graph embeddings.
Ranked #1 on
Node Classification
on USA Air-Traffic
Multi-Task Cross-Lingual Sequence Tagging from Scratch
We present a deep hierarchical recurrent neural network for sequence tagging.
Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs
GenVector leverages large-scale unlabeled data with embeddings and represents data of two modalities---i. e., social network users and knowledge concepts---in a shared latent topic space.
