Search Results for author:
Found 22 papers, 13 papers with code
Unsupervised Domain Adaptation for Question Generation with DomainData Selection and Self-training
We test our approaches on three large public datasets with different domain similarities, using a transformer-based pre-trained QG model.
When the Music Stops: Tip-of-the-Tongue Retrieval for Music
We present a study of Tip-of-the-tongue (ToT) retrieval for music, where a searcher is trying to find an existing music entity, but is unable to succeed as they cannot accurately recall important identifying information.
Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance Assessments
The creation of relevance assessments by human assessors (often nowadays crowdworkers) is a vital step when building IR test collections.
Perspectives on Large Language Models for Relevance Judgment
When asked, large language models (LLMs) like ChatGPT claim that they can assist with relevance judgments but it is not clear whether automated judgments can reliably be used in evaluations of retrieval systems.
Do the Findings of Document and Passage Retrieval Generalize to the Retrieval of Responses for Dialogues?
A number of learned sparse and dense retrieval approaches have recently been proposed and proven effective in tasks such as passage retrieval and document retrieval.
Users and Contemporary SERPs: A (Re-)Investigation Examining User Interactions and Experiences
The Search Engine Results Page (SERP) has evolved significantly over the last two decades, moving away from the simple ten blue links paradigm to considerably more complex presentations that contain results from multiple verticals and granularities of textual information.
Moving Stuff Around: A study on efficiency of moving documents into memory for Neural IR models
We show that, when using the most popular libraries for neural ranker research (i. e. PyTorch and Hugging Face's Transformers), the practice of loading all documents into main memory is not always the fastest option and is not feasible for setups with more than a couple GPUs.
Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialogues
Ranking responses for a given dialogue context is a popular benchmark in which the setup is to re-rank the ground-truth response over a limited set of $n$ responses, where $n$ is typically 10.
Searching, Learning, and Subtopic Ordering: A Simulation-based Analysis
Complex search tasks - such as those from the Search as Learning (SAL) domain - often result in users developing an information need composed of several aspects.
Diagnosing BERT with Retrieval Heuristics
This means that the axiomatic approach to IR (and its extension of diagnostic datasets created for retrieval heuristics) may in its current form not be applicable to large-scale corpora.
Searching to Learn with Instructional Scaffolding
Search engines are considered the primary tool to assist and empower learners in finding information relevant to their learning goals-be it learning something new, improving their existing skills, or just fulfilling a curiosity.
Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators
Our experimental results across two datasets for two IR tasks reveal that retrieval pipelines are not robust to these query variations, with effectiveness drops of $\approx20\%$ on average.
On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search
According to the Probability Ranking Principle (PRP), ranking documents in decreasing order of their probability of relevance leads to an optimal document ranking for ad-hoc retrieval.
On the Calibration and Uncertainty of Neural Learning to Rank Models
Our experimental results on the ad-hoc retrieval task of conversation response ranking reveal that (i) BERT-based rankers are not robustly calibrated and that stochastic BERT-based rankers yield better calibration; and (ii) uncertainty estimation is beneficial for both risk-aware neural ranking, i. e. taking into account the uncertainty when ranking documents, and for predicting unanswerable conversational contexts.
Weakly Supervised Label Smoothing
Inspired by our investigation of LS in the context of neural L2R models, we propose a novel technique called Weakly Supervised Label Smoothing (WSLS) that takes advantage of the retrieval scores of the negative sampled documents as a weak supervision signal in the process of modifying the ground-truth labels.
Slice-Aware Neural Ranking
Understanding when and why neural ranking models fail for an IR task via error analysis is an important part of the research cycle.
What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
Overall, our analyses and experiments show that: (i) BERT has knowledge stored in its parameters about the content of books, movies and music; (ii) it has more content-based knowledge than collaborative-based knowledge; and (iii) fails on conversational recommendation when faced with adversarial data.
Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
Curriculum learning has recently been shown to improve neural models' effectiveness by sampling batches non-uniformly, going from easy to difficult instances during training.
Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset
Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs.
Feature Engineering for Second Language Acquisition Modeling
Knowledge tracing serves as a keystone in delivering personalized education.
