Search Results for author: Claudia Hauff

Found 22 papers, 13 papers with code

When the Music Stops: Tip-of-the-Tongue Retrieval for Music

1 code implementation23 May 2023 Samarth Bhargav, Anne Schuth, Claudia Hauff

We present a study of Tip-of-the-tongue (ToT) retrieval for music, where a searcher is trying to find an existing music entity, but is unable to succeed as they cannot accurately recall important identifying information.

Benchmarking Language Modelling +3

Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance Assessments

no code implementations21 Apr 2023 Nirmal Roy, Agathe Balayn, David Maxwell, Claudia Hauff

The creation of relevance assessments by human assessors (often nowadays crowdworkers) is a vital step when building IR test collections.

Perspectives on Large Language Models for Relevance Judgment

no code implementations13 Apr 2023 Guglielmo Faggioli, Laura Dietz, Charles Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, Henning Wachsmuth

When asked, large language models (LLMs) like ChatGPT claim that they can assist with relevance judgments but it is not clear whether automated judgments can reliably be used in evaluations of retrieval systems.


Do the Findings of Document and Passage Retrieval Generalize to the Retrieval of Responses for Dialogues?

1 code implementation13 Jan 2023 Gustavo Penha, Claudia Hauff

A number of learned sparse and dense retrieval approaches have recently been proposed and proven effective in tasks such as passage retrieval and document retrieval.

Conversational Search Passage Retrieval +1

Users and Contemporary SERPs: A (Re-)Investigation Examining User Interactions and Experiences

no code implementations26 Jul 2022 Nirmal Roy, David Maxwell, Claudia Hauff

The Search Engine Results Page (SERP) has evolved significantly over the last two decades, moving away from the simple ten blue links paradigm to considerably more complex presentations that contain results from multiple verticals and granularities of textual information.

Moving Stuff Around: A study on efficiency of moving documents into memory for Neural IR models

1 code implementation17 May 2022 Arthur Câmara, Claudia Hauff

We show that, when using the most popular libraries for neural ranker research (i. e. PyTorch and Hugging Face's Transformers), the practice of loading all documents into main memory is not always the fastest option and is not feasible for setups with more than a couple GPUs.

Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialogues

1 code implementation22 Apr 2022 Gustavo Penha, Claudia Hauff

Ranking responses for a given dialogue context is a popular benchmark in which the setup is to re-rank the ground-truth response over a limited set of $n$ responses, where $n$ is typically 10.

Language Modelling Retrieval +1

Searching, Learning, and Subtopic Ordering: A Simulation-based Analysis

no code implementations26 Jan 2022 Arthur Câmara, David Maxwell, Claudia Hauff

Complex search tasks - such as those from the Search as Learning (SAL) domain - often result in users developing an information need composed of several aspects.

Diagnosing BERT with Retrieval Heuristics

1 code implementation12 Jan 2022 Arthur Câmara, Claudia Hauff

This means that the axiomatic approach to IR (and its extension of diagnostic datasets created for retrieval heuristics) may in its current form not be applicable to large-scale corpora.

Information Retrieval Retrieval +1

Searching to Learn with Instructional Scaffolding

1 code implementation29 Nov 2021 Arthur Câmara, Nirmal Roy, David Maxwell, Claudia Hauff

Search engines are considered the primary tool to assist and empower learners in finding information relevant to their learning goals-be it learning something new, improving their existing skills, or just fulfilling a curiosity.

Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators

1 code implementation25 Nov 2021 Gustavo Penha, Arthur Câmara, Claudia Hauff

Our experimental results across two datasets for two IR tasks reveal that retrieval pipelines are not robust to these query variations, with effectiveness drops of $\approx20\%$ on average.

Information Retrieval Language Modelling +1

On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search

no code implementations EACL 2021 Gustavo Penha, Claudia Hauff

According to the Probability Ranking Principle (PRP), ranking documents in decreasing order of their probability of relevance leads to an optimal document ranking for ad-hoc retrieval.

Conversational Search Document Ranking +2

On the Calibration and Uncertainty of Neural Learning to Rank Models

1 code implementation12 Jan 2021 Gustavo Penha, Claudia Hauff

Our experimental results on the ad-hoc retrieval task of conversation response ranking reveal that (i) BERT-based rankers are not robustly calibrated and that stochastic BERT-based rankers yield better calibration; and (ii) uncertainty estimation is beneficial for both risk-aware neural ranking, i. e. taking into account the uncertainty when ranking documents, and for predicting unanswerable conversational contexts.

Document Ranking Learning-To-Rank +1

Weakly Supervised Label Smoothing

1 code implementation15 Dec 2020 Gustavo Penha, Claudia Hauff

Inspired by our investigation of LS in the context of neural L2R models, we propose a novel technique called Weakly Supervised Label Smoothing (WSLS) that takes advantage of the retrieval scores of the negative sampled documents as a weak supervision signal in the process of modifying the ground-truth labels.

Learning-To-Rank Passage Retrieval +1

Slice-Aware Neural Ranking

1 code implementation EMNLP (scai) 2020 Gustavo Penha, Claudia Hauff

Understanding when and why neural ranking models fail for an IR task via error analysis is an important part of the research cycle.

What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation

1 code implementation30 Jul 2020 Gustavo Penha, Claudia Hauff

Overall, our analyses and experiments show that: (i) BERT has knowledge stored in its parameters about the content of books, movies and music; (ii) it has more content-based knowledge than collaborative-based knowledge; and (iii) fails on conversational recommendation when faced with adversarial data.

Language Modelling Recommendation Systems +1

Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking

1 code implementation18 Dec 2019 Gustavo Penha, Claudia Hauff

Curriculum learning has recently been shown to improve neural models' effectiveness by sampling batches non-uniformly, going from easy to difficult instances during training.

Information Retrieval Retrieval

Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset

2 code implementations10 Dec 2019 Gustavo Penha, Alexandru Balan, Claudia Hauff

Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs.

Conversational Search Information Retrieval +1

Cannot find the paper you are looking for? You can Submit a new open access paper.