Search Results for author: Gustavo Penha

Found 7 papers, 5 papers with code

On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search

no code implementations EACL 2021 Gustavo Penha, Claudia Hauff

According to the Probability Ranking Principle (PRP), ranking documents in decreasing order of their probability of relevance leads to an optimal document ranking for ad-hoc retrieval.

Conversational Search Document Ranking +1

On the Calibration and Uncertainty of Neural Learning to Rank Models

1 code implementation12 Jan 2021 Gustavo Penha, Claudia Hauff

Our experimental results on the ad-hoc retrieval task of conversation response ranking reveal that (i) BERT-based rankers are not robustly calibrated and that stochastic BERT-based rankers yield better calibration; and (ii) uncertainty estimation is beneficial for both risk-aware neural ranking, i. e. taking into account the uncertainty when ranking documents, and for predicting unanswerable conversational contexts.

Document Ranking Learning-To-Rank

Weakly Supervised Label Smoothing

no code implementations15 Dec 2020 Gustavo Penha, Claudia Hauff

Inspired by our investigation of LS in the context of neural L2R models, we propose a novel technique called Weakly Supervised Label Smoothing (WSLS) that takes advantage of the retrieval scores of the negative sampled documents as a weak supervision signal in the process of modifying the ground-truth labels.

Curriculum Learning Learning-To-Rank +1

Slice-Aware Neural Ranking

1 code implementation7 Oct 2020 Gustavo Penha, Claudia Hauff

Understanding when and why neural ranking models fail for an IR task via error analysis is an important part of the research cycle.

What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation

1 code implementation30 Jul 2020 Gustavo Penha, Claudia Hauff

Overall, our analyses and experiments show that: (i) BERT has knowledge stored in its parameters about the content of books, movies and music; (ii) it has more content-based knowledge than collaborative-based knowledge; and (iii) fails on conversational recommendation when faced with adversarial data.

Language Modelling Recommendation Systems

Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking

1 code implementation18 Dec 2019 Gustavo Penha, Claudia Hauff

Curriculum learning has recently been shown to improve neural models' effectiveness by sampling batches non-uniformly, going from easy to difficult instances during training.

Curriculum Learning Information Retrieval

Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset

1 code implementation10 Dec 2019 Gustavo Penha, Alexandru Balan, Claudia Hauff

Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs.

Conversational Search Information Retrieval +1

Cannot find the paper you are looking for? You can Submit a new open access paper.