Search Results for author:
Found 127 papers, 60 papers with code
Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering
Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer.
Improving Language Plasticity via Pretraining with Active Forgetting
Pretrained language models (PLMs) are today the primary model for natural language processing.
Can discrete information extraction prompts generalize across language models?
We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information.
Task-aware Retrieval with Instructions
We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries.
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks
Experiments on various knowledge-intensive tasks such as question answering and dialogue datasets show that, simply augmenting parametric models (T5-base) using our method produces more accurate results (e. g., 25. 8 -> 44. 3 EM on NQ) while retaining a high throughput (e. g., 1000 queries/s on NQ).
Ranked #4 on
Question Answering
on KILT: ELI5
Query Expansion Using Contextual Clue Sampling with Language Models
To balance these two considerations, we propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context.
EditEval: An Instruction-Based Benchmark for Text Improvements
Evaluation of text generation to date has primarily focused on content created sequentially, rather than improvements on a piece of text.
PEER: A Collaborative Language Model
Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes.
Atlas: Few-shot Learning with Retrieval Augmented Language Models
Retrieval augmented models are known to excel at knowledge intensive tasks without the need for as many parameters, but it is unclear whether they work in few-shot settings.
Ranked #1 on
Question Answering
on Natural Questions
ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective
Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs).
Improving Wikipedia Verifiability with AI
Hence, maintaining and improving the quality of Wikipedia references is an important challenge and there is a pressing need for better tools to assist humans in this effort.
EDIN: An End-to-end Benchmark and Pipeline for Unknown Entity Discovery and Indexing
This paper created the Unknown Entity Discovery and Indexing (EDIN) benchmark where unknown entities, that is entities without a description in the knowledge base and labeled mentions, have to be integrated into an existing entity linking system.
Lifting the Curse of Multilinguality by Pre-training Modular Transformers
Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages.
Open Vocabulary Extreme Classification Using Generative Models
To develop systems that simplify this process, we introduce the task of open vocabulary XMC (OXMC): given a piece of content, predict a set of labels, some of which may be outside of the known tag set.
Autoregressive Search Engines: Generating Substrings as Document Identifiers
Knowledge-intensive language tasks require NLP systems to both provide the correct answer and retrieve supporting evidence for it in a given corpus.
The Web Is Your Oyster -- Knowledge-Intensive NLP against a Very Large Web Corpus
In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-domain environment: web-scale knowledge, lack of structure, inconsistent quality and noise.
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants
We collect training datasets in twenty experimental settings and perform a detailed analysis of this approach for the task of extractive question answering (QA) for both standard and adversarial data collection.
Unsupervised Dense Information Retrieval with Contrastive Learning
In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings.
Boosted Dense Retriever
DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on retrieval mistakes made by the current ensemble.
A Few More Examples May Be Worth Billions of Parameters
We investigate the dynamics of increasing the number of model parameters versus the number of labeled examples across a wide variety of tasks.
Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations
Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC).
Ranked #1 on
Link Prediction
on CoDEx Small
Contrastive Pre-training for Zero-Shot Information Retrieval
By contrast, in many other NLP tasks, conventional self-supervised pre-training based on masking leads to strong generalization with small number of training examples.
Challenges in Generalization in Open Domain Question Answering
Recent work on Open Domain Question Answering has shown that there is a large discrepancy in model performance between novel test questions and those that largely overlap with training questions.
ProoFVer: Natural Logic Theorem Proving for Fact Verification
Fact verification systems typically rely on neural network classifiers for veracity prediction which lack explainability.
Ranked #1 on
Fact Verification
on FEVER
Domain-matched Pre-training Tasks for Dense Retrieval
Pre-training on larger datasets with ever increasing model size is now a proven recipe for increased performance across almost all NLP tasks.
Ranked #2 on
Passage Retrieval
on Natural Questions
(using extra training data)
Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints
Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering (ODQA) systems.
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning.
Database Reasoning Over Text
Neural models have shown impressive performance gains in answering queries from natural language text.
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
When primed with only a handful of training samples, very large, pretrained language models such as GPT-3 have shown competitive results when compared to fully-supervised, fine-tuned, large, pretrained language models.
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation
We further conduct a novel human-in-the-loop evaluation to show that our models are considerably more robust to new human-written adversarial examples: crowdworkers can fool our model only 8. 8% of the time on average, compared to 17. 6% for a model trained without synthetic data.
Dynabench: Rethinking Benchmarking in NLP
We introduce Dynabench, an open-source platform for dynamic dataset creation and model benchmarking.
Multilingual Autoregressive Entity Linking
Moreover, in a zero-shot setting on languages with no training data at all, mGENRE treats the target language as a latent variable that is marginalized at prediction time.
Ranked #2 on
Entity Disambiguation
on Mewsli-9
(using extra training data)
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them
We introduce a new QA-pair retriever, RePAQ, to complement PAQ.
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020.
A Memory Efficient Baseline for Open Domain Question Answering
Recently, retrieval systems based on dense representations have led to important improvements in open-domain question answering, and related tasks.
Joint Verification and Reranking for Open Fact Checking Over Tables
Structured information is an important knowledge source for automatic verification of factual claims.
Generating Fact Checking Briefs
Fact checking at scale is difficult -- while the number of active fact checking websites is growing, it remains too small for the needs of the contemporary media ecosystem.
Don't Read Too Much into It: Adaptive Computation for Open-Domain Question Answering
Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer.
Neural Databases
We describe NeuralDB, a database system with no pre-defined schema, in which updates and queries are given in natural language.
Autoregressive Entity Retrieval
For instance, Encyclopedias such as Wikipedia are structured by entities (e. g., one per Wikipedia article).
Ranked #1 on
Entity Linking
on Derczynski
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.
Ranked #16 on
Question Answering
on HotpotQA
KILT: a Benchmark for Knowledge Intensive Language Tasks
We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
Ranked #3 on
Entity Linking
on KILT: WNED-CWEB
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
We also find that 30% of test-set questions have a near-duplicate paraphrase in their corresponding training sets.
Learning Reasoning Strategies in End-to-End Differentiable Proving
Attempts to render deep learning models interpretable, data-efficient, and robust have seen some success through hybridisation with rule-based systems, for example, in Neural Theorem Provers (NTPs).
Ranked #1 on
Relational Reasoning
on CLUTRR (k=3)
Concept Matching for Low-Resource Classification
We propose a model to tackle classification tasks in the presence of very little training data.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
Ranked #4 on
Question Answering
on WebQuestions
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks.
Ranked #6 on
Text-To-SQL
on spider
How Context Affects Language Models' Factual Predictions
When pre-trained on large unsupervised textual corpora, language models are able to store and retrieve factual knowledge to some extent, making it possible to use them directly for zero-shot cloze-style question answering.
AxCell: Automatic Extraction of Results from Machine Learning Papers
Tracking progress in machine learning has become increasingly difficult with the recent explosion in the number of papers.
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training
Natural Language Inference (NLI) datasets contain annotation artefacts resulting in spurious correlations between the natural language utterances and their respective entailment classes.
Undersensitivity in Neural Reading Comprehension
Current reading comprehension models generalise well to in-distribution test sets, yet perform poorly on adversarially selected inputs.
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension
We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop.
Ranked #1 on
Reading Comprehension
on AdversarialQA
(using extra training data)
Differentiable Reasoning on Large Knowledge Bases and Natural Language
Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.
Ranked #3 on
Link Prediction
on FB122
Scalable Zero-shot Entity Linking with Dense Entity Retrieval
This paper introduces a conceptually simple, scalable, and highly effective BERT-based entity linking model, along with an extensive evaluation of its accuracy-speed trade-off.
How Decoding Strategies Affect the Verifiability of Generated Text
A generated sentence is verifiable if it can be corroborated or disproved by Wikipedia, and we find that the verifiability of generated text strongly depends on the decoding strategy.
MLQA: Evaluating Cross-lingual Extractive Question Answering
An alternative to building large monolingual training datasets is to develop cross-lingual systems which can transfer to a target language without requiring training data in that language.
MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actions
This is largely an artifact of building on top of frameworks designed for RL in games (e. g. OpenAI Gym).
Soft Token Matching for Interpretable Low-Resource Classification
We propose a model to tackle classification tasks in the presence of very little training data.
Language Models as Knowledge Bases?
Recent progress in pretraining language models on large textual corpora led to a surge of improvements for downstream NLP tasks.
Comparing Semi-Parametric Model Learning Algorithms for Dynamic Model Estimation in Robotics
Physical modeling of robotic system behavior is the foundation for controlling many robotic mechanisms to a satisfactory degree.
Unsupervised Question Answering by Cloze Translation
We approach this problem by first learning to generate context, question and answer triples in an unsupervised manner, which we then use to synthesize Extractive QA training data automatically.
Neural Variational Inference For Estimating Uncertainty in Knowledge Graph Embeddings
Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data.
Scalable Neural Theorem Proving on Knowledge Bases and Natural Language
Reasoning over text and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.
Evaluating Rewards for Question Generation Models
Recent approaches to question generation have used modifications to a Seq2Seq architecture inspired by advances in machine translation.
Ranked #12 on
Question Generation
on SQuAD1.1
Towards Machine-assisted Meta-Studies: The Hubble Constant
We present an approach for automatic extraction of measured values from the astrophysical literature, using the Hubble constant for our pilot study.
Application of Clinical Concept Embeddings for Heart Failure Prediction in UK EHR data
Electronic health records (EHR) are increasingly being used for constructing disease risk prediction models.
UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)
In this paper we describe our 2nd place FEVER shared-task system that achieved a FEVER score of 62. 52{\%} on the provisional test set (without additional human evaluation), and 65. 41{\%} on the development set.
Assumption Questioning: Latent Copying and Reward Exploitation in Question Generation
Question generation is an important task for improving our ability to process natural language data, with additional challenges over other sequence transformation tasks.
Logical Rule Induction and Theory Learning Using Neural Theorem Proving
Our approach is neuro-symbolic in the sense that the rule pred- icates and core facts are given dense vector representations.
Interpretation of Natural Language Rules in Conversational Machine Reading
This task requires both the interpretation of rules and the application of background knowledge.
Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge
They are useful for understanding the shortcomings of machine learning models, interpreting their results, and for regularisation.
Towards Neural Theorem Proving at Scale
Neural models combining representation learning and reasoning in an end-to-end trainable manner are receiving increasing interest.
Jack the Reader -- A Machine Reading Framework
For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.
Jack the Reader - A Machine Reading Framework
For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.
Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers
In this paper, we explore different strategies for modelling numerals with language models, such as memorisation and digit-by-digit composition, and propose a novel neural architecture that uses a continuous probability density function to model numerals from an open vocabulary.
Extrapolation in NLP
We argue that extrapolation to examples outside the training space will often be easier for models that capture global structures, rather than just maximise their local fit to the training data.
Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness
Natural Language Inference is a challenging task that has received substantial attention, and state-of-the-art models now achieve impressive test set performance in the form of accuracy scores.
Reduce, Reuse, Recycle: New uses for old QA resources
We investigate applying repurposed generic QA data and models to a recently proposed relation extraction task.
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
We propose a novel task to encourage the development of models for text understanding across multiple documents and to investigate the limits of existing methods.
Ranked #9 on
Question Answering
on WikiHop
Adversarial Sets for Regularising Neural Link Predictors
The training objective is defined as a minimax problem, where an adversary finds the most offending adversarial examples by maximising the inconsistency loss, and the model is trained by jointly minimising a supervised loss and the inconsistency loss on the adversarial examples.
A simple but tough-to-beat baseline for the Fake News Challenge stance detection task
Identifying public misinformation is a complicated and challenging task.
Ranked #5 on
Fake News Detection
on FNC-1
Convolutional 2D Knowledge Graph Embeddings
In this work, we introduce ConvE, a multi-layer convolutional network model for link prediction, and report state-of-the-art results for several established datasets.
Ranked #1 on
Link Prediction
on WN18
A Supervised Approach to Extractive Summarisation of Scientific Papers
Automatic summarisation is a popular approach to reduce a document to its main arguments.
End-to-End Differentiable Proving
We introduce neural networks for end-to-end differentiable proving of queries to knowledge bases by operating on dense vector representations of symbols.
SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications
We describe the SemEval task of extracting keyphrases and relations between them from scientific documents, which is crucial for understanding which publications describe which processes, tasks and materials.
The SUMMA Platform Prototype
We present the first prototype of the SUMMA Platform: an integrated platform for multilingual media monitoring.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
How Well Can We Predict Hypernyms from Word Embeddings? A Dataset-Centric Analysis
One key property of word embeddings currently under study is their capacity to encode hypernymy.
Imitation learning for structured prediction in natural language processing
Imitation learning is a learning paradigm originally developed to learn robotic controllers from demonstrations by humans, e. g. autonomous flight from pilot demonstrations.
Knowledge Graph Completion via Complex Tensor Factorization
In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predicting missing relationships---labeled edges.
Ranked #2 on
Knowledge Graphs
on FB15k
Frustratingly Short Attention Spans in Neural Language Modeling
This vector is used both for predicting the next token as well as for the key and value of a differentiable memory of a token history.
Community Question Answering Platforms vs. Twitter for Predicting Characteristics of Urban Neighbourhoods
We use the text from QA platform of Yahoo!
Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies
In this paper we present a novel Neural Network algorithm for conducting semi-supervised learning for sequence labeling tasks arranged in a linguistically motivated hierarchy.
Learning Python Code Suggestion with a Sparse Pointer Network
By augmenting a neural language model with a pointer network specialized in referring to predefined classes of identifiers, we obtain a much lower perplexity and a 5 percentage points increase in accuracy for code suggestion compared to an LSTM baseline.
Represent, Aggregate, and Constrain: A Novel Architecture for Machine Reading from Noisy Sources
In order to extract event information from text, a machine reading model must learn to accurately read and interpret the ways in which that information is expressed.
Learning to Reason With Adaptive Computation
Multi-hop inference is necessary for machine learning systems to successfully solve tasks such as Recognising Textual Entailment and Machine Reading.
Clinical Text Prediction with Numerically Grounded Conditional Language Models
In this paper, we investigate how grounded and conditional extensions to standard neural language models can bring improvements in the tasks of word prediction and completion.
SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods
In this paper, we introduce the task of targeted aspect-based sentiment analysis.
Ranked #5 on
Aspect-Based Sentiment Analysis (ABSA)
on Sentihood
emoji2vec: Learning Emoji Representations from their Description
Many current natural language processing applications for social media rely on representation learning and utilize pre-trained word embeddings.
Numerically Grounded Language Models for Semantic Error Correction
Semantic error detection and correction is an important task for applications such as fact checking, speech-to-text or grammatical error correction.
Lifted Rule Injection for Relation Embeddings
Methods based on representation learning currently hold the state-of-the-art in many natural language processing and knowledge base inference tasks.
Complex Embeddings for Simple Link Prediction
In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases.
Ranked #4 on
Link Prediction
on FB122
Neural Architectures for Fine-grained Entity Type Classification
In this work, we investigate several neural network architectures for fine-grained entity type classification.
Generating Natural Language Inference Chains
We take entailment-pairs of the Stanford Natural Language Inference corpus and train an LSTM with attention.
Programming with a Differentiable Forth Interpreter
Given that in practice training data is scarce for all but a small set of problems, a core question is how to incorporate prior knowledge into a model.
A Factorization Machine Framework for Testing Bigram Embeddings in Knowledgebase Completion
Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links.
An Attentive Neural Architecture for Fine-grained Entity Type Classification
In this work we propose a novel attention-based neural network model for the task of fine-grained entity type classification that unlike previously proposed models recursively composes representations of entity mention contexts.
Extraction of evidence tables from abstracts of randomized clinical trials using a maximum entropy classifier and global constraints
An evidence table has columns for the patient group, for each of the interventions being compared, for the criterion for the comparison (e. g. proportion who survived after 5 years from treatment), and for each of the results.
Anytime Belief Propagation Using Sparse Domains
Belief Propagation has been widely used for marginal inference, however it is slow on problems with large-domain variables and high-order factors.
Automorphism Groups of Graphical Models and Lifted Variational Inference
This automorphism group provides a precise mathematical framework for lifted inference in the general exponential family.
MAP Inference in Chains using Column Generation
Linear chains and trees are basic building blocks in many applications of graphical models.
