no code implementations • 21 Mar 2023 • Gustavo Penha, Enrico Palumbo, Maryam Aziz, Alice Wang, Hugues Bouchard
A pre-requisite to discover an entity, e. g. a book, with a search engine is that the entity is retrievable, i. e. there are queries for which the system will surface such entity in the top results.
1 code implementation • 13 Jan 2023 • Gustavo Penha, Claudia Hauff
A number of learned sparse and dense retrieval approaches have recently been proposed and proven effective in tasks such as passage retrieval and document retrieval.
1 code implementation • 22 Apr 2022 • Gustavo Penha, Claudia Hauff
Ranking responses for a given dialogue context is a popular benchmark in which the setup is to re-rank the ground-truth response over a limited set of $n$ responses, where $n$ is typically 10.
1 code implementation • 25 Nov 2021 • Gustavo Penha, Arthur Câmara, Claudia Hauff
Our experimental results across two datasets for two IR tasks reveal that retrieval pipelines are not robust to these query variations, with effectiveness drops of $\approx20\%$ on average.
no code implementations • EACL 2021 • Gustavo Penha, Claudia Hauff
According to the Probability Ranking Principle (PRP), ranking documents in decreasing order of their probability of relevance leads to an optimal document ranking for ad-hoc retrieval.
1 code implementation • 12 Jan 2021 • Gustavo Penha, Claudia Hauff
Our experimental results on the ad-hoc retrieval task of conversation response ranking reveal that (i) BERT-based rankers are not robustly calibrated and that stochastic BERT-based rankers yield better calibration; and (ii) uncertainty estimation is beneficial for both risk-aware neural ranking, i. e. taking into account the uncertainty when ranking documents, and for predicting unanswerable conversational contexts.
1 code implementation • 15 Dec 2020 • Gustavo Penha, Claudia Hauff
Inspired by our investigation of LS in the context of neural L2R models, we propose a novel technique called Weakly Supervised Label Smoothing (WSLS) that takes advantage of the retrieval scores of the negative sampled documents as a weak supervision signal in the process of modifying the ground-truth labels.
1 code implementation • EMNLP (scai) 2020 • Gustavo Penha, Claudia Hauff
Understanding when and why neural ranking models fail for an IR task via error analysis is an important part of the research cycle.
1 code implementation • 30 Jul 2020 • Gustavo Penha, Claudia Hauff
Overall, our analyses and experiments show that: (i) BERT has knowledge stored in its parameters about the content of books, movies and music; (ii) it has more content-based knowledge than collaborative-based knowledge; and (iii) fails on conversational recommendation when faced with adversarial data.
1 code implementation • 18 Dec 2019 • Gustavo Penha, Claudia Hauff
Curriculum learning has recently been shown to improve neural models' effectiveness by sampling batches non-uniformly, going from easy to difficult instances during training.
2 code implementations • 10 Dec 2019 • Gustavo Penha, Alexandru Balan, Claudia Hauff
Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs.