Search Results for author:
Found 12 papers, 7 papers with code
Circling Back to Recurrent Models of Language
Just because some purely recurrent models suffer from being hard to optimize and inefficient on today's hardware, they are not necessarily bad models of language.
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization
In practice, with a finite number of optimization steps and a learning rate that cannot be annealed to zero, Tail Averaging can get much closer to a local minimum point of the training loss than either the individual iterates or the Polyak average.
Mutual Information Constraints for Monte-Carlo Objectives
A common failure mode of density models trained as variational autoencoders is to model the data without relying on their latent variables, rendering these variables useless.
Capturing document context inside sentence-level neural machine translation models with self-training
Neural machine translation (NMT) has arguably achieved human level parity when trained and evaluated at the sentence-level.
A Critical Analysis of Biased Parsers in Unsupervised Parsing
A series of recent papers has used a parsing algorithm due to Shen et al. (2018) to recover phrase-structure trees based on proxies for "syntactic depth."
Mogrifier LSTM
Many advances in Natural Language Processing have been based upon more expressive models for how inputs interact with the context in which they occur.
Unsupervised Recurrent Neural Network Grammars
On language modeling, unsupervised RNNGs perform as well their supervised counterparts on benchmarks in English and Chinese.
Ranked #8 on
Constituency Grammar Induction
on PTB Diagnostic ECG Database
(Max F1 (WSJ) metric)
Encoding Spatial Relations from Natural Language
Natural language processing has made significant inroads into learning the semantics of words through distributional approaches, however representations learnt via these methods fail to capture certain kinds of information implicit in the real world.
Pushing the bounds of dropout
We show that dropout training is best understood as performing MAP estimation concurrently for a family of conditional models whose objectives are themselves lower bounded by the original dropout objective.
Ranked #24 on
Language Modelling
on Penn Treebank (Word Level)
The NarrativeQA Reading Comprehension Challenge
Reading comprehension (RC)---in contrast to information retrieval---requires integrating information and reasoning about events, entities, and their relations across a full document.
Ranked #9 on
Question Answering
on NarrativeQA
(BLEU-1 metric)
On the State of the Art of Evaluation in Neural Language Models
Ongoing innovations in recurrent neural network architectures have provided a steady influx of apparently state-of-the-art results on language modelling benchmarks.
Ranked #32 on
Language Modelling
on WikiText-2
Semantic Parsing with Semi-Supervised Sequential Autoencoders
We present a novel semi-supervised approach for sequence transduction and apply it to semantic parsing.
