Search Results for author: Sebastian Riedel

Found 127 papers, 60 papers with code

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering

no code implementations EMNLP (sustainlp) 2020 Yuxiang Wu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer.

Open-Domain Question Answering

Can discrete information extraction prompts generalize across language models?

1 code implementation20 Feb 2023 Nathanaël Carraz Rakotonirina, Roberto Dessì, Fabio Petroni, Sebastian Riedel, Marco Baroni

We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information.

Language Modelling slot-filling +1

Task-aware Retrieval with Instructions

1 code implementation16 Nov 2022 Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, Wen-tau Yih

We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries.


An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks

1 code implementation30 Oct 2022 Yuxiang Wu, Yu Zhao, Baotian Hu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

Experiments on various knowledge-intensive tasks such as question answering and dialogue datasets show that, simply augmenting parametric models (T5-base) using our method produces more accurate results (e. g., 25. 8 -> 44. 3 EM on NQ) while retaining a high throughput (e. g., 1000 queries/s on NQ).

Question Answering Retrieval

Query Expansion Using Contextual Clue Sampling with Language Models

no code implementations13 Oct 2022 Linqing Liu, Minghan Li, Jimmy Lin, Sebastian Riedel, Pontus Stenetorp

To balance these two considerations, we propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context.

Information Retrieval Language Modelling +1

EditEval: An Instruction-Based Benchmark for Text Improvements

1 code implementation27 Sep 2022 Jane Dwivedi-Yu, Timo Schick, Zhengbao Jiang, Maria Lomeli, Patrick Lewis, Gautier Izacard, Edouard Grave, Sebastian Riedel, Fabio Petroni

Evaluation of text generation to date has primarily focused on content created sequentially, rather than improvements on a piece of text.

Text Generation

PEER: A Collaborative Language Model

no code implementations24 Aug 2022 Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, Sebastian Riedel

Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes.

Language Modelling

Atlas: Few-shot Learning with Retrieval Augmented Language Models

1 code implementation5 Aug 2022 Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, Edouard Grave

Retrieval augmented models are known to excel at knowledge intensive tasks without the need for as many parameters, but it is unclear whether they work in few-shot settings.

Fact Checking Few-Shot Learning +6

ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

no code implementations20 Jul 2022 Yihong Chen, Pushkar Mishra, Luca Franceschi, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs).

Knowledge Graph Completion

Improving Wikipedia Verifiability with AI

1 code implementation8 Jul 2022 Fabio Petroni, Samuel Broscheit, Aleksandra Piktus, Patrick Lewis, Gautier Izacard, Lucas Hosseini, Jane Dwivedi-Yu, Maria Lomeli, Timo Schick, Pierre-Emmanuel Mazaré, Armand Joulin, Edouard Grave, Sebastian Riedel

Hence, maintaining and improving the quality of Wikipedia references is an important challenge and there is a pressing need for better tools to assist humans in this effort.

Citation Recommendation Fact Checking

EDIN: An End-to-end Benchmark and Pipeline for Unknown Entity Discovery and Indexing

1 code implementation25 May 2022 Nora Kassner, Fabio Petroni, Mikhail Plekhanov, Sebastian Riedel, Nicola Cancedda

This paper created the Unknown Entity Discovery and Indexing (EDIN) benchmark where unknown entities, that is entities without a description in the knowledge base and labeled mentions, have to be integrated into an existing entity linking system.

Entity Linking Novel Concepts +1

Lifting the Curse of Multilinguality by Pre-training Modular Transformers

no code implementations NAACL 2022 Jonas Pfeiffer, Naman Goyal, Xi Victoria Lin, Xian Li, James Cross, Sebastian Riedel, Mikel Artetxe

Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages.

named-entity-recognition Named Entity Recognition +3

Open Vocabulary Extreme Classification Using Generative Models

no code implementations Findings (ACL) 2022 Daniel Simig, Fabio Petroni, Pouya Yanki, Kashyap Popat, Christina Du, Sebastian Riedel, Majid Yazdani

To develop systems that simplify this process, we introduce the task of open vocabulary XMC (OXMC): given a piece of content, predict a set of labels, some of which may be outside of the known tag set.

Classification Extreme Multi-Label Classification +2

Autoregressive Search Engines: Generating Substrings as Document Identifiers

1 code implementation22 Apr 2022 Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen-tau Yih, Sebastian Riedel, Fabio Petroni

Knowledge-intensive language tasks require NLP systems to both provide the correct answer and retrieve supporting evidence for it in a given corpus.

Information Retrieval Retrieval

The Web Is Your Oyster -- Knowledge-Intensive NLP against a Very Large Web Corpus

2 code implementations18 Dec 2021 Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izacard, Patrick Lewis, Barlas Oğuz, Edouard Grave, Wen-tau Yih, Sebastian Riedel

In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-domain environment: web-scale knowledge, lack of structure, inconsistent quality and noise.

Common Sense Reasoning Retrieval

Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants

no code implementations NAACL 2022 Max Bartolo, Tristan Thrush, Sebastian Riedel, Pontus Stenetorp, Robin Jia, Douwe Kiela

We collect training datasets in twenty experimental settings and perform a detailed analysis of this approach for the task of extractive question answering (QA) for both standard and adversarial data collection.

Extractive Question-Answering Question Answering

Unsupervised Dense Information Retrieval with Contrastive Learning

5 code implementations16 Dec 2021 Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, Edouard Grave

In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings.

Contrastive Learning Cross-Lingual Transfer +4

Boosted Dense Retriever

no code implementations NAACL 2022 Patrick Lewis, Barlas Oğuz, Wenhan Xiong, Fabio Petroni, Wen-tau Yih, Sebastian Riedel

DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on retrieval mistakes made by the current ensemble.

Quantization Retrieval

A Few More Examples May Be Worth Billions of Parameters

1 code implementation8 Oct 2021 Yuval Kirstain, Patrick Lewis, Sebastian Riedel, Omer Levy

We investigate the dynamics of increasing the number of model parameters versus the number of labeled examples across a wide variety of tasks.

Extractive Question-Answering Multiple-choice +1

Contrastive Pre-training for Zero-Shot Information Retrieval

no code implementations29 Sep 2021 Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, Edouard Grave

By contrast, in many other NLP tasks, conventional self-supervised pre-training based on masking leads to strong generalization with small number of training examples.

Contrastive Learning Fact Checking +3

Challenges in Generalization in Open Domain Question Answering

1 code implementation Findings (NAACL) 2022 Linqing Liu, Patrick Lewis, Sebastian Riedel, Pontus Stenetorp

Recent work on Open Domain Question Answering has shown that there is a large discrepancy in model performance between novel test questions and those that largely overlap with training questions.

Natural Questions Open-Domain Question Answering +3

ProoFVer: Natural Logic Theorem Proving for Fact Verification

1 code implementation25 Aug 2021 Amrith Krishna, Sebastian Riedel, Andreas Vlachos

Fact verification systems typically rely on neural network classifiers for veracity prediction which lack explainability.

Automated Theorem Proving Decision Making +2

Database Reasoning Over Text

1 code implementation ACL 2021 James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy

Neural models have shown impressive performance gains in answering queries from natural language text.

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

1 code implementation ACL 2022 Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, Pontus Stenetorp

When primed with only a handful of training samples, very large, pretrained language models such as GPT-3 have shown competitive results when compared to fully-supervised, fine-tuned, large, pretrained language models.

text-classification Text Classification

Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation

no code implementations EMNLP 2021 Max Bartolo, Tristan Thrush, Robin Jia, Sebastian Riedel, Pontus Stenetorp, Douwe Kiela

We further conduct a novel human-in-the-loop evaluation to show that our models are considerably more robust to new human-written adversarial examples: crowdworkers can fool our model only 8. 8% of the time on average, compared to 17. 6% for a model trained without synthetic data.

Answer Selection Question Generation

Multilingual Autoregressive Entity Linking

1 code implementation23 Mar 2021 Nicola De Cao, Ledell Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke Zettlemoyer, Nicola Cancedda, Sebastian Riedel, Fabio Petroni

Moreover, in a zero-shot setting on languages with no training data at all, mGENRE treats the target language as a latent variable that is marginalized at prediction time.

Ranked #2 on Entity Disambiguation on Mewsli-9 (using extra training data)

Entity Disambiguation Entity Linking

A Memory Efficient Baseline for Open Domain Question Answering

1 code implementation30 Dec 2020 Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Sebastian Riedel, Edouard Grave

Recently, retrieval systems based on dense representations have led to important improvements in open-domain question answering, and related tasks.

Dimensionality Reduction Open-Domain Question Answering +3

Generating Fact Checking Briefs

no code implementations EMNLP 2020 Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, Sebastian Riedel

Fact checking at scale is difficult -- while the number of active fact checking websites is growing, it remains too small for the needs of the contemporary media ecosystem.

Fact Checking Question Answering

Don't Read Too Much into It: Adaptive Computation for Open-Domain Question Answering

no code implementations EMNLP 2020 Yuxiang Wu, Sebastian Riedel, Pasquale Minervini, Pontus Stenetorp

Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer.

Open-Domain Question Answering

Neural Databases

no code implementations14 Oct 2020 James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy

We describe NeuralDB, a database system with no pre-defined schema, in which updates and queries are given in natural language.


Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

1 code implementation ICLR 2021 Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen-tau Yih, Sebastian Riedel, Douwe Kiela, Barlas Oğuz

We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.

Question Answering Retrieval

Learning Reasoning Strategies in End-to-End Differentiable Proving

2 code implementations ICML 2020 Pasquale Minervini, Sebastian Riedel, Pontus Stenetorp, Edward Grefenstette, Tim Rocktäschel

Attempts to render deep learning models interpretable, data-efficient, and robust have seen some success through hybridisation with rule-based systems, for example, in Neural Theorem Provers (NTPs).

Link Prediction Relational Reasoning

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

1 code implementation ACL 2020 Pengcheng Yin, Graham Neubig, Wen-tau Yih, Sebastian Riedel

Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks.

Semantic Parsing Text-To-SQL

How Context Affects Language Models' Factual Predictions

no code implementations AKBC 2020 Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, Sebastian Riedel

When pre-trained on large unsupervised textual corpora, language models are able to store and retrieve factual knowledge to some extent, making it possible to use them directly for zero-shot cloze-style question answering.

Information Retrieval Language Modelling +3

Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training

1 code implementation EMNLP 2020 Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel, Tim Rocktäschel

Natural Language Inference (NLI) datasets contain annotation artefacts resulting in spurious correlations between the natural language utterances and their respective entailment classes.

Natural Language Inference

Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension

1 code implementation2 Feb 2020 Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, Pontus Stenetorp

We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop.

 Ranked #1 on Reading Comprehension on AdversarialQA (using extra training data)

Reading Comprehension

Differentiable Reasoning on Large Knowledge Bases and Natural Language

3 code implementations17 Dec 2019 Pasquale Minervini, Matko Bošnjak, Tim Rocktäschel, Sebastian Riedel, Edward Grefenstette

Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.

Link Prediction Question Answering +1

Scalable Zero-shot Entity Linking with Dense Entity Retrieval

3 code implementations EMNLP 2020 Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, Luke Zettlemoyer

This paper introduces a conceptually simple, scalable, and highly effective BERT-based entity linking model, along with an extensive evaluation of its accuracy-speed trade-off.

Entity Embeddings Entity Linking +3

MLQA: Evaluating Cross-lingual Extractive Question Answering

4 code implementations ACL 2020 Patrick Lewis, Barlas Oğuz, Ruty Rinott, Sebastian Riedel, Holger Schwenk

An alternative to building large monolingual training datasets is to develop cross-lingual systems which can transfer to a target language without requiring training data in that language.

Extractive Question-Answering Machine Translation +1

Comparing Semi-Parametric Model Learning Algorithms for Dynamic Model Estimation in Robotics

no code implementations27 Jun 2019 Sebastian Riedel, Freek Stulp

Physical modeling of robotic system behavior is the foundation for controlling many robotic mechanisms to a satisfactory degree.

BIG-bench Machine Learning regression

Unsupervised Question Answering by Cloze Translation

1 code implementation ACL 2019 Patrick Lewis, Ludovic Denoyer, Sebastian Riedel

We approach this problem by first learning to generate context, question and answer triples in an unsupervised manner, which we then use to synthesize Extractive QA training data automatically.

Natural Questions NMT +2

Neural Variational Inference For Estimating Uncertainty in Knowledge Graph Embeddings

1 code implementation12 Jun 2019 Alexander I. Cowen-Rivers, Pasquale Minervini, Tim Rocktaschel, Matko Bosnjak, Sebastian Riedel, Jun Wang

Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data.

Knowledge Graph Embeddings Knowledge Graphs +2

Scalable Neural Theorem Proving on Knowledge Bases and Natural Language

no code implementations ICLR 2019 Pasquale Minervini, Matko Bosnjak, Tim Rocktäschel, Edward Grefenstette, Sebastian Riedel

Reasoning over text and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.

Automated Theorem Proving Link Prediction +2

Evaluating Rewards for Question Generation Models

1 code implementation NAACL 2019 Tom Hosking, Sebastian Riedel

Recent approaches to question generation have used modifications to a Seq2Seq architecture inspired by advances in machine translation.

Machine Translation Policy Gradient Methods +3

Towards Machine-assisted Meta-Studies: The Hubble Constant

no code implementations31 Jan 2019 Tom Crossland, Pontus Stenetorp, Sebastian Riedel, Daisuke Kawata, Thomas D. Kitching, Rupert A. C. Croft

We present an approach for automatic extraction of measured values from the astrophysical literature, using the Hubble constant for our pilot study.

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

no code implementations WS 2018 Takuma Yoneda, Jeff Mitchell, Johannes Welbl, Pontus Stenetorp, Sebastian Riedel

In this paper we describe our 2nd place FEVER shared-task system that achieved a FEVER score of 62. 52{\%} on the provisional test set (without additional human evaluation), and 65. 41{\%} on the development set.

Information Retrieval Natural Language Inference +2

Assumption Questioning: Latent Copying and Reward Exploitation in Question Generation

no code implementations27 Sep 2018 Tom Hosking, Sebastian Riedel

Question generation is an important task for improving our ability to process natural language data, with additional challenges over other sequence transformation tasks.

Inductive Bias Machine Translation +4

Logical Rule Induction and Theory Learning Using Neural Theorem Proving

no code implementations6 Sep 2018 Andres Campero, Aldo Pareja, Tim Klinger, Josh Tenenbaum, Sebastian Riedel

Our approach is neuro-symbolic in the sense that the rule pred- icates and core facts are given dense vector representations.

Automated Theorem Proving

Towards Neural Theorem Proving at Scale

no code implementations21 Jul 2018 Pasquale Minervini, Matko Bosnjak, Tim Rocktäschel, Sebastian Riedel

Neural models combining representation learning and reasoning in an end-to-end trainable manner are receiving increasing interest.

Automated Theorem Proving Representation Learning

Jack the Reader -- A Machine Reading Framework

1 code implementation ACL 2018 Dirk Weissenborn, Pasquale Minervini, Isabelle Augenstein, Johannes Welbl, Tim Rockt{\"a}schel, Matko Bo{\v{s}}njak, Jeff Mitchell, Thomas Demeester, Tim Dettmers, Pontus Stenetorp, Sebastian Riedel

For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.

Information Retrieval Link Prediction +4

Jack the Reader - A Machine Reading Framework

2 code implementations20 Jun 2018 Dirk Weissenborn, Pasquale Minervini, Tim Dettmers, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bošnjak, Jeff Mitchell, Thomas Demeester, Pontus Stenetorp, Sebastian Riedel

For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.

Link Prediction Natural Language Inference +3

Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers

1 code implementation ACL 2018 Georgios P. Spithourakis, Sebastian Riedel

In this paper, we explore different strategies for modelling numerals with language models, such as memorisation and digit-by-digit composition, and propose a novel neural architecture that uses a continuous probability density function to model numerals from an open vocabulary.

Language Modelling

Extrapolation in NLP

no code implementations WS 2018 Jeff Mitchell, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

We argue that extrapolation to examples outside the training space will often be easier for models that capture global structures, rather than just maximise their local fit to the training data.

Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness

no code implementations NAACL 2018 Vicente Ivan Sanchez Carmona, Jeff Mitchell, Sebastian Riedel

Natural Language Inference is a challenging task that has received substantial attention, and state-of-the-art models now achieve impressive test set performance in the form of accuracy scores.

Natural Language Inference

Reduce, Reuse, Recycle: New uses for old QA resources

no code implementations22 Apr 2018 Jeff Mitchell, Sebastian Riedel

We investigate applying repurposed generic QA data and models to a recently proposed relation extraction task.

Relation Extraction slot-filling +1

Constructing Datasets for Multi-hop Reading Comprehension Across Documents

no code implementations TACL 2018 Johannes Welbl, Pontus Stenetorp, Sebastian Riedel

We propose a novel task to encourage the development of models for text understanding across multiple documents and to investigate the limits of existing methods.

Multi-Hop Reading Comprehension

Adversarial Sets for Regularising Neural Link Predictors

1 code implementation24 Jul 2017 Pasquale Minervini, Thomas Demeester, Tim Rocktäschel, Sebastian Riedel

The training objective is defined as a minimax problem, where an adversary finds the most offending adversarial examples by maximising the inconsistency loss, and the model is trained by jointly minimising a supervised loss and the inconsistency loss on the adversarial examples.

Link Prediction Relational Reasoning

Convolutional 2D Knowledge Graph Embeddings

8 code implementations5 Jul 2017 Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

In this work, we introduce ConvE, a multi-layer convolutional network model for link prediction, and report state-of-the-art results for several established datasets.

Knowledge Graph Embeddings Knowledge Graphs +1

End-to-End Differentiable Proving

3 code implementations NeurIPS 2017 Tim Rocktäschel, Sebastian Riedel

We introduce neural networks for end-to-end differentiable proving of queries to knowledge bases by operating on dense vector representations of symbols.

Link Prediction

SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications

1 code implementation SEMEVAL 2017 Isabelle Augenstein, Mrinal Das, Sebastian Riedel, Lakshmi Vikraman, Andrew McCallum

We describe the SemEval task of extracting keyphrases and relations between them from scientific documents, which is crucial for understanding which publications describe which processes, tasks and materials.

Knowledge Base Population

Imitation learning for structured prediction in natural language processing

no code implementations EACL 2017 Andreas Vlachos, Gerasimos Lampouras, Sebastian Riedel

Imitation learning is a learning paradigm originally developed to learn robotic controllers from demonstrations by humans, e. g. autonomous flight from pilot demonstrations.

coreference-resolution Dependency Parsing +5

Knowledge Graph Completion via Complex Tensor Factorization

2 code implementations22 Feb 2017 Théo Trouillon, Christopher R. Dance, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard

In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predicting missing relationships---labeled edges.

Link Prediction Relational Reasoning

Frustratingly Short Attention Spans in Neural Language Modeling

no code implementations15 Feb 2017 Michał Daniluk, Tim Rocktäschel, Johannes Welbl, Sebastian Riedel

This vector is used both for predicting the next token as well as for the key and value of a differentiable memory of a token history.

Language Modelling

Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies

no code implementations29 Dec 2016 Jonathan Godwin, Pontus Stenetorp, Sebastian Riedel

In this paper we present a novel Neural Network algorithm for conducting semi-supervised learning for sequence labeling tasks arranged in a linguistically motivated hierarchy.


Learning Python Code Suggestion with a Sparse Pointer Network

5 code implementations24 Nov 2016 Avishkar Bhoopchand, Tim Rocktäschel, Earl Barr, Sebastian Riedel

By augmenting a neural language model with a pointer network specialized in referring to predefined classes of identifiers, we obtain a much lower perplexity and a 5 percentage points increase in accuracy for code suggestion compared to an LSTM baseline.

Language Modelling

Represent, Aggregate, and Constrain: A Novel Architecture for Machine Reading from Noisy Sources

no code implementations30 Oct 2016 Jason Naradowsky, Sebastian Riedel

In order to extract event information from text, a machine reading model must learn to accurately read and interpret the ways in which that information is expressed.

Reading Comprehension

Learning to Reason With Adaptive Computation

no code implementations24 Oct 2016 Mark Neumann, Pontus Stenetorp, Sebastian Riedel

Multi-hop inference is necessary for machine learning systems to successfully solve tasks such as Recognising Textual Entailment and Machine Reading.

BIG-bench Machine Learning Natural Language Inference +1

Clinical Text Prediction with Numerically Grounded Conditional Language Models

no code implementations WS 2016 Georgios P. Spithourakis, Steffen E. Petersen, Sebastian Riedel

In this paper, we investigate how grounded and conditional extensions to standard neural language models can bring improvements in the tasks of word prediction and completion.

emoji2vec: Learning Emoji Representations from their Description

7 code implementations WS 2016 Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak, Sebastian Riedel

Many current natural language processing applications for social media rely on representation learning and utilize pre-trained word embeddings.

Representation Learning Sentiment Analysis +1

Numerically Grounded Language Models for Semantic Error Correction

no code implementations EMNLP 2016 Georgios P. Spithourakis, Isabelle Augenstein, Sebastian Riedel

Semantic error detection and correction is an important task for applications such as fact checking, speech-to-text or grammatical error correction.

Fact Checking Grammatical Error Correction +1

Lifted Rule Injection for Relation Embeddings

no code implementations EMNLP 2016 Thomas Demeester, Tim Rocktäschel, Sebastian Riedel

Methods based on representation learning currently hold the state-of-the-art in many natural language processing and knowledge base inference tasks.

Representation Learning

Complex Embeddings for Simple Link Prediction

8 code implementations20 Jun 2016 Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard

In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases.

Link Prediction Relational Reasoning

Generating Natural Language Inference Chains

no code implementations4 Jun 2016 Vladyslav Kolesnyk, Tim Rocktäschel, Sebastian Riedel

We take entailment-pairs of the Stanford Natural Language Inference corpus and train an LSTM with attention.

Machine Translation Natural Language Inference +1

Programming with a Differentiable Forth Interpreter

1 code implementation ICML 2017 Matko Bošnjak, Tim Rocktäschel, Jason Naradowsky, Sebastian Riedel

Given that in practice training data is scarce for all but a small set of problems, a core question is how to incorporate prior knowledge into a model.

A Factorization Machine Framework for Testing Bigram Embeddings in Knowledgebase Completion

no code implementations WS 2016 Johannes Welbl, Guillaume Bouchard, Sebastian Riedel

Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links.

Knowledge Base Completion

An Attentive Neural Architecture for Fine-grained Entity Type Classification

no code implementations WS 2016 Sonse Shimaoka, Pontus Stenetorp, Kentaro Inui, Sebastian Riedel

In this work we propose a novel attention-based neural network model for the task of fine-grained entity type classification that unlike previously proposed models recursively composes representations of entity mention contexts.

Classification General Classification +1

Extraction of evidence tables from abstracts of randomized clinical trials using a maximum entropy classifier and global constraints

2 code implementations17 Sep 2015 Antonio Trenta, Anthony Hunter, Sebastian Riedel

An evidence table has columns for the patient group, for each of the interventions being compared, for the criterion for the comparison (e. g. proportion who survived after 5 years from treatment), and for each of the results.

Anytime Belief Propagation Using Sparse Domains

no code implementations14 Nov 2013 Sameer Singh, Sebastian Riedel, Andrew McCallum

Belief Propagation has been widely used for marginal inference, however it is slow on problems with large-domain variables and high-order factors.


Automorphism Groups of Graphical Models and Lifted Variational Inference

no code implementations26 Sep 2013 Hung Bui, Tuyen Huynh, Sebastian Riedel

This automorphism group provides a precise mathematical framework for lifted inference in the general exponential family.

Variational Inference

Cannot find the paper you are looking for? You can Submit a new open access paper.