Search Results for author: Johannes Welbl

Found 20 papers, 8 papers with code

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

no code implementations NA 2021 Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent SIfre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, Geoffrey Irving

Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.

Fact Checking Language Modelling +2

Evaluating the Apperception Engine

no code implementations9 Jul 2020 Richard Evans, Jose Hernandez-Orallo, Johannes Welbl, Pushmeet Kohli, Marek Sergot

This is notable because our system is not a bespoke system designed specifically to solve intelligence tests, but a general-purpose system that was designed to make sense of any sensory sequence.

Inductive logic programming Unity

Towards Verified Robustness under Text Deletion Interventions

no code implementations ICLR 2020 Johannes Welbl, Po-Sen Huang, Robert Stanforth, Sven Gowal, Krishnamurthy (Dj) Dvijotham, Martin Szummer, Pushmeet Kohli

Neural networks are widely used in Natural Language Processing, yet despite their empirical successes, their behaviour is brittle: they are both over-sensitive to small input changes, and under-sensitive to deletions of large fractions of input text.

Natural Language Inference

Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension

1 code implementation2 Feb 2020 Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, Pontus Stenetorp

We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop.

 Ranked #1 on Reading Comprehension on AdversarialQA (using extra training data)

Reading Comprehension

Making sense of sensory input

1 code implementation5 Oct 2019 Richard Evans, Jose Hernandez-Orallo, Johannes Welbl, Pushmeet Kohli, Marek Sergot

This is notable because our system is not a bespoke system designed specifically to solve intelligence tests, but a general-purpose system that was designed to make sense of any sensory sequence.

Program Synthesis Unity

Scalable Neural Learning for Verifiable Consistency with Temporal Specifications

no code implementations25 Sep 2019 Sumanth Dathathri, Johannes Welbl, Krishnamurthy (Dj) Dvijotham, Ramana Kumar, Aditya Kanade, Jonathan Uesato, Sven Gowal, Po-Sen Huang, Pushmeet Kohli

Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to small perturbations of the input features.

Adversarial Robustness Language Modelling

UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

no code implementations WS 2018 Takuma Yoneda, Jeff Mitchell, Johannes Welbl, Pontus Stenetorp, Sebastian Riedel

In this paper we describe our 2nd place FEVER shared-task system that achieved a FEVER score of 62. 52{\%} on the provisional test set (without additional human evaluation), and 65. 41{\%} on the development set.

Information Retrieval Natural Language Inference +1

Jack the Reader -- A Machine Reading Framework

1 code implementation ACL 2018 Dirk Weissenborn, Pasquale Minervini, Isabelle Augenstein, Johannes Welbl, Tim Rockt{\"a}schel, Matko Bo{\v{s}}njak, Jeff Mitchell, Thomas Demeester, Tim Dettmers, Pontus Stenetorp, Sebastian Riedel

For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.

Information Retrieval Link Prediction +4

Jack the Reader - A Machine Reading Framework

2 code implementations20 Jun 2018 Dirk Weissenborn, Pasquale Minervini, Tim Dettmers, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bošnjak, Jeff Mitchell, Thomas Demeester, Pontus Stenetorp, Sebastian Riedel

For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.

Link Prediction Natural Language Inference +3

Constructing Datasets for Multi-hop Reading Comprehension Across Documents

no code implementations TACL 2018 Johannes Welbl, Pontus Stenetorp, Sebastian Riedel

We propose a novel task to encourage the development of models for text understanding across multiple documents and to investigate the limits of existing methods.

Multi-Hop Reading Comprehension

Crowdsourcing Multiple Choice Science Questions

no code implementations WS 2017 Johannes Welbl, Nelson F. Liu, Matt Gardner

With this method we have assembled SciQ, a dataset of 13. 7K multiple choice science exam questions (Dataset available at http://allenai. org/data. html).

Question Generation

Knowledge Graph Completion via Complex Tensor Factorization

2 code implementations22 Feb 2017 Théo Trouillon, Christopher R. Dance, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard

In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predicting missing relationships---labeled edges.

Knowledge Graph Completion Link Prediction +1

Frustratingly Short Attention Spans in Neural Language Modeling

no code implementations15 Feb 2017 Michał Daniluk, Tim Rocktäschel, Johannes Welbl, Sebastian Riedel

This vector is used both for predicting the next token as well as for the key and value of a differentiable memory of a token history.

Language Modelling

Complex Embeddings for Simple Link Prediction

4 code implementations20 Jun 2016 Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard

In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases.

Link Prediction Relational Reasoning

Neural Random Forests

2 code implementations25 Apr 2016 Gérard Biau, Erwan Scornet, Johannes Welbl

Given an ensemble of randomized regression trees, it is possible to restructure them as a collection of multilayered neural networks with particular connection weights.

A Factorization Machine Framework for Testing Bigram Embeddings in Knowledgebase Completion

no code implementations WS 2016 Johannes Welbl, Guillaume Bouchard, Sebastian Riedel

Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links.

Knowledge Base Completion

Cannot find the paper you are looking for? You can Submit a new open access paper.