Search Results for author: Pasquale Minervini

Found 58 papers, 31 papers with code

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering

no code implementations EMNLP (sustainlp) 2020 Yuxiang Wu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer.

Open-Domain Question Answering

Conditional computation in neural networks: principles and research trends

no code implementations12 Mar 2024 Simone Scardapane, Alessandro Baiocchi, Alessio Devoto, Valerio Marsocci, Pasquale Minervini, Jary Pomponi

This article summarizes principles and ideas from the emerging area of applying \textit{conditional computation} methods to the design of neural networks.

Transfer Learning

FairBelief - Assessing Harmful Beliefs in Language Models

no code implementations27 Feb 2024 Mattia Setzu, Marta Marchiori Manerba, Pasquale Minervini, Debora Nozza

Language Models (LMs) have been shown to inherit undesired biases that might hurt minorities and underrepresented groups if such systems were integrated into real-world applications without careful fairness auditing.

Fairness

Analysing The Impact of Sequence Composition on Language Model Pre-Training

1 code implementation21 Feb 2024 Yu Zhao, Yuanbin Qu, Konrad Staniszewski, Szymon Tworkowski, Wei Liu, Piotr Miłoś, Yuxiang Wu, Pasquale Minervini

In this work, we find that applying causal masking can lead to the inclusion of distracting information from previous documents during pre-training, which negatively impacts the performance of the models on language modelling and downstream tasks.

In-Context Learning Language Modelling +1

Using Natural Language Explanations to Improve Robustness of In-context Learning for Natural Language Inference

no code implementations13 Nov 2023 Xuanli He, Yuxiang Wu, Oana-Maria Camburu, Pasquale Minervini, Pontus Stenetorp

Moreover, we introduce a new approach to X-ICL by prompting an LLM (ChatGPT in our case) with few human-generated NLEs to produce further NLEs (we call it ChatGPT few-shot), which we show superior to both ChatGPT zero-shot and human-generated NLEs alone.

In-Context Learning Natural Language Inference

REFER: An End-to-end Rationale Extraction Framework for Explanation Regularization

no code implementations22 Oct 2023 Mohammad Reza Ghasemi Madani, Pasquale Minervini

We analyze the impact of using human highlights during training by jointly training the task model and the rationale extractor.

Temporal Smoothness Regularisers for Neural Link Predictors

no code implementations16 Sep 2023 Manuel Dileo, Pasquale Minervini, Matteo Zignani, Sabrina Gaito

Furthermore, we evaluate the impact of a wide range of temporal smoothing regularisers on two state-of-the-art temporal link prediction models.

Knowledge Graphs Link Prediction +2

Approximate Answering of Graph Queries

no code implementations12 Aug 2023 Michael Cochez, Dimitrios Alivanistos, Erik Arakelyan, Max Berrendorf, Daniel Daza, Mikhail Galkin, Pasquale Minervini, Mathias Niepert, Hongyu Ren

We will first provide an overview of the different query types which can be supported by these methods and datasets typically used for evaluation, as well as an insight into their limitations.

Knowledge Graphs World Knowledge

Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain

no code implementations6 Jul 2023 Aryo Pradipta Gema, Luke Daines, Pasquale Minervini, Beatrice Alex

Additionally, we propose a two-step PEFT framework which fuses Clinical LLaMA-LoRA with Downstream LLaMA-LoRA, another PEFT adapter specialised for downstream tasks.

Domain Adaptation

Logical Reasoning for Natural Language Inference Using Generated Facts as Atoms

no code implementations22 May 2023 Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Oana-Maria Camburu, Marek Rei

We apply our method to the highly challenging ANLI dataset, where our framework improves the performance of both a DeBERTa-base and BERT baseline.

Logical Reasoning Natural Language Inference +1

Machine Learning-Assisted Recurrence Prediction for Early-Stage Non-Small-Cell Lung Cancer Patients

no code implementations17 Nov 2022 Adrianna Janik, Maria Torrente, Luca Costabello, Virginia Calvo, Brian Walsh, Carlos Camps, Sameh K. Mohamed, Ana L. Ortega, Vít Nováček, Bartomeu Massutí, Pasquale Minervini, M. Rosario Garcia Campelo, Edel del Barco, Joaquim Bosch-Barrera, Ernestina Menasalvas, Mohan Timilsina, Mariano Provencio

Conclusions: Our results show that machine learning models trained on tabular and graph data can enable objective, personalised and reproducible prediction of relapse and therefore, disease outcome in patients with early-stage NSCLC.

An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks

1 code implementation30 Oct 2022 Yuxiang Wu, Yu Zhao, Baotian Hu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

Experiments on various knowledge-intensive tasks such as question answering and dialogue datasets show that, simply augmenting parametric models (T5-base) using our method produces more accurate results (e. g., 25. 8 -> 44. 3 EM on NQ) while retaining a high throughput (e. g., 1000 queries/s on NQ).

Computational Efficiency Question Answering +1

Learning Discrete Directed Acyclic Graphs via Backpropagation

no code implementations27 Oct 2022 Andrew J. Wren, Pasquale Minervini, Luca Franceschi, Valentina Zantedeschi

Recently continuous relaxations have been proposed in order to learn Directed Acyclic Graphs (DAGs) from data by backpropagation, instead of using combinatorial optimization.

Combinatorial Optimization

Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models

1 code implementation11 Sep 2022 Pasquale Minervini, Luca Franceschi, Mathias Niepert

In this work, we present Adaptive IMLE (AIMLE), the first adaptive gradient estimator for complex discrete distributions: it adaptively identifies the target distribution for IMLE by trading off the density of gradient information with the degree of bias in the gradient estimates.

ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

no code implementations20 Jul 2022 Yihong Chen, Pushkar Mishra, Luca Franceschi, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs).

Knowledge Graph Completion

XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking

1 code implementation12 Apr 2022 Han Zhou, Ignacio Iacobacci, Pasquale Minervini

Dialogue State Tracking (DST), a crucial component of task-oriented dialogue (ToD) systems, keeps track of all important information pertaining to dialogue history: filling slots with the most probable values throughout the conversation.

Cross-Lingual Transfer Dialogue State Tracking +4

Differentiable Reasoning over Long Stories -- Assessing Systematic Generalisation in Neural Models

no code implementations20 Mar 2022 Wanshui Li, Pasquale Minervini

Contemporary neural networks have achieved a series of developments and successes in many aspects; however, when exposed to data outside the training distribution, they may fail to predict correct answers.

Natural Language Understanding

A Probabilistic Framework for Knowledge Graph Data Augmentation

2 code implementations25 Oct 2021 Jatin Chauhan, Priyanshu Gupta, Pasquale Minervini

We present NNMFAug, a probabilistic framework to perform data augmentation for the task of knowledge graph completion to counter the problem of data scarcity, which can enhance the learning process of neural link predictors.

Data Augmentation Knowledge Graph Completion +1

Neuro-Symbolic Ontology-Mediated Query Answering

no code implementations29 Sep 2021 Medina Andresel, Daria Stepanova, Trung-Kien Tran, Csaba Domokos, Pasquale Minervini

Recently, low-dimensional vector space representations of Knowledge Graphs (KGs) have been applied to find answers to logical queries over incomplete KGs.

Data Augmentation Knowledge Graphs

Combining Inductive and Deductive Reasoning for Query Answering over Incomplete Knowledge Graphs

1 code implementation26 Jun 2021 Medina Andresel, Trung-Kien Tran, Csaba Domokos, Pasquale Minervini, Daria Stepanova

Current methods for embedding-based query answering over incomplete Knowledge Graphs (KGs) only focus on inductive reasoning, i. e., predicting answers by learning patterns from the data, and lack the complementary ability to do deductive reasoning, which requires the application of domain knowledge to infer further information.

Data Augmentation Knowledge Graphs

Neural Concept Formation in Knowledge Graphs

1 code implementation AKBC 2021 Agnieszka Dobrowolska, Antonio Vergari, Pasquale Minervini

In this work, we investigate how to learn novel concepts in Knowledge Graphs (KGs) in a principled way, and how to effectively exploit them to produce more accurate neural link prediction models.

Knowledge Graphs Link Prediction +1

Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions

2 code implementations NeurIPS 2021 Mathias Niepert, Pasquale Minervini, Luca Franceschi

We propose Implicit Maximum Likelihood Estimation (I-MLE), a framework for end-to-end learning of models combining discrete exponential family distributions and differentiable neural components.

Combinatorial Optimization

Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning

1 code implementation8 Feb 2021 Zhengyao Jiang, Pasquale Minervini, Minqi Jiang, Tim Rocktaschel

In this work, we show that we can incorporate relational inductive biases, encoded in the form of relational graphs, into agents.

reinforcement-learning Reinforcement Learning (RL)

Don't Read Too Much into It: Adaptive Computation for Open-Domain Question Answering

no code implementations EMNLP 2020 Yuxiang Wu, Sebastian Riedel, Pasquale Minervini, Pontus Stenetorp

Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer.

Open-Domain Question Answering

Complex Query Answering with Neural Link Predictors

3 code implementations ICLR 2021 Erik Arakelyan, Daniel Daza, Pasquale Minervini, Michael Cochez

Finally, we demonstrate that it is possible to explain the outcome of our model in terms of the intermediate solutions identified for each of the complex query atoms.

Complex Query Answering

WordCraft: An Environment for Benchmarking Commonsense Agents

1 code implementation ICML Workshop LaReL 2020 Minqi Jiang, Jelena Luketina, Nantas Nardelli, Pasquale Minervini, Philip H. S. Torr, Shimon Whiteson, Tim Rocktäschel

This is partly due to the lack of lightweight simulation environments that sufficiently reflect the semantics of the real world and provide knowledge sources grounded with respect to observations in an RL environment.

Benchmarking Knowledge Graphs +2

Learning Reasoning Strategies in End-to-End Differentiable Proving

2 code implementations ICML 2020 Pasquale Minervini, Sebastian Riedel, Pontus Stenetorp, Edward Grefenstette, Tim Rocktäschel

Attempts to render deep learning models interpretable, data-efficient, and robust have seen some success through hybridisation with rule-based systems, for example, in Neural Theorem Provers (NTPs).

Link Prediction Relational Reasoning

Knowledge Graph Embeddings and Explainable AI

no code implementations30 Apr 2020 Federico Bianchi, Gaetano Rossiello, Luca Costabello, Matteo Palmonari, Pasquale Minervini

Knowledge graph embeddings are now a widely adopted approach to knowledge representation in which entities and relationships are embedded in vector spaces.

Knowledge Graph Embeddings

Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training

1 code implementation EMNLP 2020 Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel, Tim Rocktäschel

Natural Language Inference (NLI) datasets contain annotation artefacts resulting in spurious correlations between the natural language utterances and their respective entailment classes.

Natural Language Inference Sentence

Differentiable Reasoning on Large Knowledge Bases and Natural Language

3 code implementations17 Dec 2019 Pasquale Minervini, Matko Bošnjak, Tim Rocktäschel, Sebastian Riedel, Edward Grefenstette

Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.

Link Prediction Question Answering +1

Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations

1 code implementation ACL 2020 Oana-Maria Camburu, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz, Phil Blunsom

To increase trust in artificial intelligence systems, a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions.

Decision Making Natural Language Inference

NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language

1 code implementation ACL 2019 Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, Tim Rocktäschel

In contrast, neural models can cope very well with ambiguity by learning distributed representations of words and their composition from data, but lead to models that are difficult to interpret.

Question Answering Sentence

Neural Variational Inference For Estimating Uncertainty in Knowledge Graph Embeddings

1 code implementation12 Jun 2019 Alexander I. Cowen-Rivers, Pasquale Minervini, Tim Rocktaschel, Matko Bosnjak, Sebastian Riedel, Jun Wang

Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data.

Knowledge Graph Embeddings Knowledge Graphs +2

Neural Variational Inference For Embedding Knowledge Graphs

no code implementations ICLR 2019 Alexander I. Cowen-Rivers, Pasquale Minervini

While traditional variational methods derive an analytical approximation for the intractable distribution over the latent variables, here we construct an inference network conditioned on the symbolic representation of entities and relation types in the Knowledge Graph, to provide the variational distributions.

Knowledge Graphs Variational Inference

NLProlog: Reasoning with Weak Unification for Natural Language Question Answering

no code implementations ICLR 2019 Leon Weber, Pasquale Minervini, Ulf Leser, Tim Rocktäschel

Currently, most work in natural language processing focuses on neural networks which learn distributed representations of words and their composition, thereby performing well in the presence of large linguistic variability.

Question Answering Sentence

Scalable Neural Theorem Proving on Knowledge Bases and Natural Language

no code implementations ICLR 2019 Pasquale Minervini, Matko Bosnjak, Tim Rocktäschel, Edward Grefenstette, Sebastian Riedel

Reasoning over text and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering.

Automated Theorem Proving Link Prediction +2

Embedding Cardinality Constraints in Neural Link Predictors

no code implementations16 Dec 2018 Emir Muñoz, Pasquale Minervini, Matthias Nickles

Neural link predictors learn distributed representations of entities and relations in a knowledge graph.

Knowledge Base Completion Link Prediction

Towards Neural Theorem Proving at Scale

no code implementations21 Jul 2018 Pasquale Minervini, Matko Bosnjak, Tim Rocktäschel, Sebastian Riedel

Neural models combining representation learning and reasoning in an end-to-end trainable manner are receiving increasing interest.

Automated Theorem Proving Representation Learning

Jack the Reader -- A Machine Reading Framework

1 code implementation ACL 2018 Dirk Weissenborn, Pasquale Minervini, Isabelle Augenstein, Johannes Welbl, Tim Rockt{\"a}schel, Matko Bo{\v{s}}njak, Jeff Mitchell, Thomas Demeester, Tim Dettmers, Pontus Stenetorp, Sebastian Riedel

For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.

Information Retrieval Link Prediction +4

Jack the Reader - A Machine Reading Framework

2 code implementations20 Jun 2018 Dirk Weissenborn, Pasquale Minervini, Tim Dettmers, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bošnjak, Jeff Mitchell, Thomas Demeester, Pontus Stenetorp, Sebastian Riedel

For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions.

Link Prediction Natural Language Inference +3

Extrapolation in NLP

no code implementations WS 2018 Jeff Mitchell, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

We argue that extrapolation to examples outside the training space will often be easier for models that capture global structures, rather than just maximise their local fit to the training data.

Adversarial Sets for Regularising Neural Link Predictors

1 code implementation24 Jul 2017 Pasquale Minervini, Thomas Demeester, Tim Rocktäschel, Sebastian Riedel

The training objective is defined as a minimax problem, where an adversary finds the most offending adversarial examples by maximising the inconsistency loss, and the model is trained by jointly minimising a supervised loss and the inconsistency loss on the adversarial examples.

Link Prediction Relational Reasoning

Convolutional 2D Knowledge Graph Embeddings

8 code implementations5 Jul 2017 Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

In this work, we introduce ConvE, a multi-layer convolutional network model for link prediction, and report state-of-the-art results for several established datasets.

 Ranked #1 on Link Prediction on WN18 (using extra training data)

Knowledge Graph Embeddings Knowledge Graphs +1

Cannot find the paper you are looking for? You can Submit a new open access paper.