Search Results for author:
Found 64 papers, 39 papers with code
VISA: Retrieval Augmented Generation with Visual Source Attribution
Generation with source attribution is important for enhancing the verifiability of retrieval-augmented generation (RAG) systems.
2D Matryoshka Training for Information Retrieval
In this reproducibility study, we implement and evaluate both versions of 2D Matryoshka Training on STS tasks and extend our analysis to retrieval tasks.
Starbucks: Improved Training for 2D Matryoshka Embeddings
Effective approaches that can scale embedding model depth (i. e. layers) and embedding size allow for the creation of models that are highly scalable across different computational resources and task requirements.
Does Vec2Text Pose a New Corpus Poisoning Threat?
In this paper, we take a new look at Vec2Text and investigate how much of a threat it poses to the different attacks of corpus poisoning, whereby an attacker injects adversarial passages into a retrieval corpus with the intention of misleading dense retrievers.
Source-Free Domain-Invariant Performance Prediction
In this work, we propose a source-free approach centred on uncertainty-based estimation, using a generative model for calibration in the absence of source data.
Embark on DenseQuest: A System for Selecting the Best Dense Retriever for a Custom Collection
In this demo we present a web-based application for selecting an effective pre-trained dense retriever to use on a private collection.
Dense Retrieval with Continuous Explicit Feedback for Systematic Review Screening Prioritisation
In this paper, we propose an alternative approach that still relies on neural models, but leverages dense representations and relevance feedback to enhance screening prioritisation, without the need for costly model fine-tuning and inference.
An Investigation of Prompt Variations for Zero-shot LLM-based Rankers
We provide a systematic understanding of the impact of specific components and wordings used in prompts on the effectiveness of rankers based on zero-shot Large Language Models (LLMs).
A Systematic Investigation of Distilling Large Language Models into Cross-Encoders for Passage Re-ranking
Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data.
PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval
Utilizing large language models (LLMs) for zero-shot document ranking is done in one of two ways: (1) prompt-based re-ranking methods, which require no further training but are only feasible for re-ranking a handful of candidate documents due to computational costs; and (2) unsupervised contrastive trained dense retrieval methods, which can retrieve relevant documents from the entire corpus but require a large amount of paired text data for contrastive training.
Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
Existing cross-encoder re-rankers can be categorized as pointwise, pairwise, or listwise models.
Understanding and Mitigating the Threat of Vec2Text to Dense Retrieval Systems
The emergence of Vec2Text -- a method for text embedding inversion -- has raised serious privacy concerns for dense retrieval systems which use text embeddings, such as those offered by OpenAI and Cohere.
FeB4RAG: Evaluating Federated Search in the Context of Retrieval Augmented Generation
Federated search systems aggregate results from multiple search engines, selecting appropriate sources to enhance result quality and align with user intent.
Large Language Models for Stemming: Promises, Pitfalls and Failures
With this respect, we identify three avenues, each characterised by different trade-offs in terms of computational cost, effectiveness and robustness : (1) use LLMs to stem the vocabulary for a collection, i. e., the set of unique words that appear in the collection (vocabulary stemming), (2) use LLMs to stem each document separately (contextual stemming), and (3) use LLMs to extract from each document entities that should not be stemmed, then use vocabulary stemming to stem the rest of the terms (entity-based contextual stemming).
Leveraging LLMs for Unsupervised Dense Retriever Ranking
In this paper we present Large Language Model Assisted Retrieval Model Ranking (LARMOR), an effective unsupervised approach that leverages LLMs for selecting which dense retriever to use on a test corpus (target).
ReSLLM: Large Language Models are Strong Resource Selectors for Federated Search
Our ReSLLM method exploits LLMs to drive the selection of resources in federated search in a zero-shot setting.
How to Forget Clients in Federated Online Learning to Rank?
In a FOLTR system, a ranker is learned by aggregating local updates to the global ranking model.
TPRF: A Transformer-based Pseudo-Relevance Feedback Model for Efficient and Effective Retrieval
This paper considers Pseudo-Relevance Feedback (PRF) methods for dense retrievers in a resource constrained environment such as that of cheap cloud instances or embedded systems (e. g., smartphones and smartwatches), where memory and CPU are limited and GPUs are not present.
A Reproducibility Study of Goldilocks: Just-Right Tuning of BERT for TAR
In this context, we show that there is no need for further pre-training if a domain-specific BERT backbone is used within the active learning pipeline.
Zero-shot Generative Large Language Models for Systematic Review Screening Automation
Systematic reviews are crucial for evidence-based medicine as they comprehensively analyse published research findings on specific questions.
Team IELAB at TREC Clinical Trial Track 2023: Enhancing Clinical Trial Retrieval with Neural Rankers and Large Language Models
We describe team ielab from CSIRO and The University of Queensland's approach to the 2023 TREC Clinical Trials Track.
Evaluating Generative Ad Hoc Information Retrieval
To lay a foundation for developing new evaluation methods for generative retrieval systems, we survey the relevant literature from the fields of information retrieval and natural language processing, identify search tasks and system architectures in generative retrieval, develop a new user model, and study its operationalization.
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking
In the field of information retrieval, Query Likelihood Models (QLMs) rank documents based on the probability of generating the query given the content of a document.
A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models
We propose a novel zero-shot document ranking approach based on Large Language Models (LLMs): the Setwise prompting approach.
Selecting which Dense Retriever to use for Zero-Shot Search
We propose the new problem of choosing which dense retrieval model to use when searching on a new collection for which no labels are available, i. e. in a zero-shot setting.
ChatGPT Hallucinates when Attributing Answers
We find that ChatGPT provides correct or partially correct answers in about half of the cases (50. 6% of the times), but its suggested references only exist 14% of the times.
Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection
We further find that gains provided by AL strategies come at the expense of more assessments (thus higher annotation costs) and AL strategies underperform random selection when comparing effectiveness given a fixed annotation cost.
Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisation
Our best approach is not only viable based on the information available at the time of screening, but also has similar effectiveness to the final title.
An Analysis of Untargeted Poisoning Attack and Defense Methods for Federated Online Learning to Rank Systems
For this, FOLTR trains learning to rank models in an online manner -- i. e. by exploiting users' interactions with the search systems (queries, clicks), rather than labels -- and federatively -- i. e. by not aggregating interaction data in a central server for training purposes, but by training instances of a model on each user device on their own private data, and then sharing the model updates, not the data, across a set of users that have formed the federation.
Outcome-based Evaluation of Systematic Review Automation
We find that accounting for the difference in review outcomes leads to a different assessment of the quality of a system than if traditional evaluation measures were used.
Exploring the Representation Power of SPLADE Models
The SPLADE (SParse Lexical AnD Expansion) model is a highly effective approach to learned sparse retrieval, where documents are represented by term impact scores derived from large language models.
Beyond CO2 Emissions: The Overlooked Impact of Water Consumption of Information Retrieval Models
As in other fields of artificial intelligence, the information retrieval community has grown interested in investigating the power consumption associated with neural models, particularly models of search.
Augmenting Passage Representations with Query Generation for Enhanced Cross-Lingual Dense Retrieval
Effective cross-lingual dense retrieval methods that rely on multilingual pre-trained language models (PLMs) need to be trained to encompass both the relevance matching task and the cross-language alignment task.
Typos-aware Bottlenecked Pre-Training for Robust Dense Retrieval
To address this challenge, we propose ToRoDer (TypOs-aware bottlenecked pre-training for RObust DEnse Retrieval), a novel re-training strategy for DRs that increases their robustness to misspelled queries while preserving their effectiveness in downstream retrieval tasks.
Dr ChatGPT, tell me what I want to hear: How prompt knowledge impacts health answer correctness
Aside from measuring the effectiveness of ChatGPT in this context, we show that the knowledge passed in the prompt can overturn the knowledge encoded in the model and this is, in our experiments, to the detriment of answer correctness.
Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?
The ability of ChatGPT to follow complex instructions and generate queries with high precision makes it a valuable tool for researchers conducting systematic reviews, particularly for rapid reviews where time is a constraint and often trading-off higher precision for lower recall is acceptable.
AgAsk: An Agent to Help Answer Farmer's Questions From Scientific Documents
On the basis of these needs we release an information retrieval test collection comprising real questions, a large collection of scientific documents split in passages, and ground truth relevance assessments indicating which passages are relevant to each question.
Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search
An empirical analysis compares how effective neural methods compare to traditional methods for this task.
MeSH Suggester: A Library and System for MeSH Term Suggestion for Systematic Review Boolean Query Construction
One challenge to creating an effective systematic review Boolean query is the selection of effective MeSH Terms to include in the query.
Guiding Neural Entity Alignment with Compatibility
Making compatible predictions thus should be one of the goals of training an EA model along with fitting the labelled data: this aspect however is neglected in current methods.
Dependency-aware Self-training for Entity Alignment
Entity Alignment (EA), which aims to detect entity mappings (i. e. equivalent entity pairs) in different Knowledge Graphs (KGs), is critical for KG fusion.
Automated MeSH Term Suggestion for Effective Query Formulation in Systematic Reviews Literature Search
However, identifying the correct MeSH terms to include in a query is difficult: information experts are often unfamiliar with the MeSH database and unsure about the appropriateness of MeSH terms for a query.
High-quality Task Division for Large-scale Entity Alignment
To include in the EA subtasks a high proportion of the potential mappings originally present in the large EA task, we devise a counterpart discovery method that exploits the locality principle of the EA task and the power of trained EA models.
Rethinking Persistent Homology for Visual Recognition
This paper performs a detailed analysis of the effectiveness of topological properties for image classification in various training scenarios, defined by: the number of training samples, the complexity of the training data and the complexity of the backbone network.
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation
This problem is further exacerbated when using DSI for cross-lingual retrieval, where document text and query text are in different languages.
How does Feedback Signal Quality Impact Effectiveness of Pseudo Relevance Feedback for Passage Retrieval?
Pseudo-Relevance Feedback (PRF) assumes that the top results retrieved by a first-stage ranker are relevant to the original query and uses them to improve the query representation for a second round of retrieval.
To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
In this paper we consider the problem of combining the relevance signals from sparse and dense retrievers in the context of Pseudo Relevance Feedback (PRF).
Is Non-IID Data a Threat in Federated Online Learning to Rank?
A well-known factor that affects the performance of federated learning systems, and that poses serious challenges to these approaches, is that there may be some type of bias in the way data is distributed across clients.
From Little Things Big Things Grow: A Collection with Seed Studies for Medical Systematic Review Literature Search
However, we show pseudo seed studies are not representative of real seed studies used by information specialists.
CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typos
We then demonstrate that the root cause of this resides in the input tokenization strategy employed by BERT.
Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach
We then exploit such historic implicit interactions to improve the effectiveness of a DR. A key challenge that we study is the effect that biases in the click signal, such as position bias, have on the DRs.
Asyncval: A Toolkit for Asynchronously Validating Dense Retriever Checkpoints during Training
A simple and efficient strategy to validate deep learning checkpoints is the addition of validation loops to execute during training.
Case law retrieval: problems, methods, challenges and evaluations in the last 20 years
Case law retrieval is the retrieval of judicial decisions relevant to a legal question.
Reinforcement Online Learning to Rank with Unbiased Reward Shaping
Online learning to rank (OLTR) aims to learn a ranker directly from implicit feedback derived from users' interactions, such as clicks.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback: A Reproducibility Study
Finally, we contribute a study of the generalisability of the ANCE-PRF method when dense retrievers other than ANCE are used for the first round of retrieval and for encoding the PRF signal.
Seed-driven Document Ranking for Systematic Reviews: A Reproducibility Study
Our results also indicate that our reproduced screening prioritisation method, (1) is generalisable across datasets of similar and different topicality compared to the original implementation, (2) that when using multiple seed studies, the effectiveness of the method increases using our techniques to enable this, (3) and that the use of multiple seed studies produces more stable rankings compared to single seed studies.
ActiveEA: Active Learning for Neural Entity Alignment
Entity Alignment (EA) aims to match equivalent entities across different Knowledge Graphs (KGs) and is an essential step of KG fusion.
Dealing with Typos for BERT-based Passage Retrieval and Ranking
Our experimental results on the MS MARCO passage ranking dataset show that, with our proposed typos-aware training, DR and BERT re-ranker can become robust to typos in queries, resulting in significantly improved effectiveness compared to models trained without appropriately accounting for typos.
Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls
Text-based PRF results show that the use of PRF had a mixed effect on deep rerankers across different datasets.
Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansion
BERT-based information retrieval models are expensive, in both time (query latency) and computational resources (energy, hardware cost), making many of these models impractical especially under resource constraints.
The Benefits of Word Embeddings Features for Active Learning in Clinical Information Extraction
This study investigates the use of unsupervised word embeddings and sequence features for sample representation in an active learning framework built to extract clinical concepts from clinical free text.
Building Evaluation Datasets for Consumer-Oriented Information Retrieval
In this paper we present the datasets created by CLEF eHealth Lab from 2013-2015 for evaluation of search solutions to support common people finding health information online.
