Search Results for author:
Found 22 papers, 1 papers with code
A Natural Bias for Language Generation Models
After just a few hundred training updates, a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, which inherently makes it difficult to estimate the right probability distribution over next tokens.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
Abstract Algebra
on BIG-bench
Ethical and social risks of harm from Language Models
We discuss the points of origin of different risks and point to potential mitigation approaches.
You should evaluate your language model on marginal likelihood over tokenisations
We suggest that this approach is unsatisfactory and may bottleneck our evaluation of language model performance.
Pretraining the Noisy Channel Model for Task-Oriented Dialogue
Direct decoding for task-oriented dialogue is known to suffer from the explaining-away effect, manifested in models that prefer short and generic responses.
Ranked #2 on
End-To-End Dialogue Modelling
on MULTIWOZ 2.0
Probing Emergent Semantics in Predictive Agents via Question Answering
Recent work has shown how predictive modeling can endow agents with rich knowledge of their surroundings, improving their ability to act in complex environments.
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Textual representation learners trained on large amounts of data have achieved notable success on downstream tasks; intriguingly, they have also performed well on challenging tests of syntactic competence.
Neural Generative Rhetorical Structure Parsing
We show that, for our parser's traversal order, previous beam search algorithms for RNNGs have a left-branching bias which is ill-suited for RST parsing.
Scalable Syntax-Aware Language Models Using Knowledge Distillation
Prior work has shown that, on small amounts of training data, syntactic neural language models learn structurally sensitive generalisations more successfully than sequential language models.
Learning to Negate Adjectives with Bilinear Models
We learn a mapping that negates adjectives by predicting an adjective{'}s antonym in an arbitrary word embedding model.
Proceedings of the 2016 Workshop on Semantic Spaces at the Intersection of NLP, Physics and Cognitive Science
This volume contains the Proceedings of the 2016 Workshop on Semantic Spaces at the Intersection of NLP, Physics and Cognitive Science (SLPCS 2016), which was held on the 11th of June at the University of Strathclyde, Glasgow, and was co-located with Quantum Physics and Logic (QPL 2016).
Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning
Recent work on word embeddings has shown that simple vector subtraction over pre-trained embeddings is surprisingly effective at capturing different lexical relations, despite lacking explicit supervision.
Using Sentence Plausibility to Learn the Semantics of Transitive Verbs
The functional approach to compositional distributional semantics considers transitive verbs to be linear maps that transform the distributional vectors representing nouns into a vector representing a sentence.
Evaluation of Simple Distributional Compositional Operations on Longer Texts
Distributional semantic models have been effective at representing linguistic semantics at the word level, and more recently research has moved on to the construction of distributional representations for larger segments of text.
