Search Results for author:
Found 40 papers, 27 papers with code
Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs
GenVector leverages large-scale unlabeled data with embeddings and represents data of two modalities---i. e., social network users and knowledge concepts---in a shared latent topic space.
Multi-Task Cross-Lingual Sequence Tagging from Scratch
We present a deep hierarchical recurrent neural network for sequence tagging.
Revisiting Semi-Supervised Learning with Graph Embeddings
We present a semi-supervised learning framework based on graph embeddings.
Ranked #2 on
Node Classification
on USA Air-Traffic
Review Networks for Caption Generation
We propose a novel extension of the encoder-decoder framework, called a review network.
Gated-Attention Readers for Text Comprehension
In this paper we study the problem of answering cloze-style questions over documents.
Ranked #1 on
Question Answering
on Children's Book Test
Words or Characters? Fine-grained Gating for Reading Comprehension
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension.
Ranked #50 on
Question Answering
on SQuAD1.1 dev
Semi-Supervised QA with Generative Domain-Adaptive Nets
In this framework, we train a generative model to generate questions based on the unlabeled text, and combine model-generated questions with human-generated questions for training question answering models.
A Probabilistic Framework for Location Inference from Social Media
In this paper, we formalize the problem of inferring location from social media into a semi-supervised factor graph model (SSFGM).
Differentiable Learning of Logical Rules for Knowledge Base Reasoning
We propose a framework, Neural Logic Programming, that combines the parameter and structure learning of first-order logical rules in an end-to-end differentiable model.
Linguistic Knowledge as Memory for Recurrent Neural Networks
We introduce a model that encodes such graphs as explicit memory in recurrent neural networks, and use it to model coreference relations in text.
Ranked #1 on
Question Answering
on CNN / Daily Mail
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks.
Ranked #10 on
Part-Of-Speech Tagging
on Penn Treebank
Good Semi-supervised Learning that Requires a Bad GAN
Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time.
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck.
Ranked #11 on
Language Modelling
on Penn Treebank (Word Level)
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent
Contrary to most natural language processing research, which makes use of static datasets, humans learn language interactively, grounded in an environment.
Neural Models for Reasoning over Multiple Mentions using Coreference
Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text.
Ranked #7 on
Question Answering
on WikiHop
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden unit), or embedding-free units such as image pixels.
Neural Cross-Lingual Named Entity Recognition with Minimal Resources
To improve robustness to word order differences, we propose to use self-attention, which allows for a degree of flexibility with respect to word order.
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers.
Ranked #34 on
Question Answering
on HotpotQA
GLoMo: Unsupervised Learning of Transferable Relational Graphs
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden units), or embedding-free units such as image pixels.
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.
Ranked #3 on
Language Modelling
on One Billion Word
XLNet: Generalized Autoregressive Pretraining for Language Understanding
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
Ranked #1 on
Question Answering
on SQuAD2.0 dev
Mixtape: Breaking the Softmax Bottleneck Efficiently
The softmax bottleneck has been shown to limit the expressiveness of neural language models.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
On a wide range of tasks across NLU, conditional and unconditional generation, GLM outperforms BERT, T5, and GPT given the same model sizes and data, and achieves the best performance from a single pretrained model with 1. 25x parameters of BERT Large , demonstrating its generalizability to different downstream tasks.
Ranked #4 on
Language Modelling
on WikiText-103
(using extra training data)
GPT Understands, Too
Prompting a pretrained language model with natural language patterns has been proved effective for natural language understanding (NLU).
Controllable Generation from Pre-trained Language Models via Inverse Prompting
To tackle this challenge, we propose an innovative method, inverse prompting, to better control text generation.
FastMoE: A Fast Mixture-of-Expert Training System
However, training trillion-scale MoE requires algorithm and system co-design for a well-tuned high performance distributed training system.
VeniBot: Towards Autonomous Venipuncture with Automatic Puncture Area and Angle Regression from NIR Images
For the automation, we focus on the positioning part and propose a Dual-In-Dual-Out network based on two-step learning and two-task learning, which can achieve fully automatic regression of the suitable puncture area and angle from near-infrared(NIR) images.
Distribution Matching for Rationalization
The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks.
FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning
Most previous methods for text data augmentation are limited to simple tasks and weak baselines.
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding
The few-shot natural language understanding (NLU) task has attracted much recent attention.
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training.
NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework
Pretrained language models have become the standard approach for many NLP tasks due to strong performance, but they are very expensive to train.
ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization
We propose a multitask pretraining approach ZeroPrompt for zero-shot generalization, focusing on task scaling and zero-shot prompting.
GPS: Genetic Prompt Search for Efficient Few-shot Learning
Prompt-based techniques have demostrated great potential for improving the few-shot generalization of pretrained language models.
Prompt-Based Metric Learning for Few-Shot NER
Few-shot named entity recognition (NER) targets generalizing to unseen labels and/or domains with few labeled examples.
Zero-Label Prompt Selection
Natural language prompts have been shown to facilitate cross-task generalization for large language models.
A Universal Discriminator for Zero-Shot Generalization
In the finetuning setting, our approach also achieves new state-of-the-art results on a wide range of NLP tasks, with only 1/4 parameters of previous methods.
Learning to Detect Noisy Labels Using Model-Based Features
Label noise is ubiquitous in various machine learning scenarios such as self-labeling with model predictions and erroneous data annotation.
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X
Large pre-trained code generation models, such as OpenAI Codex, can generate syntax- and function-correct code, making the coding of programmers more productive and our pursuit of artificial general intelligence closer.
Ranked #81 on
Code Generation
on MBPP
P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks
Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training.
