1 code implementation • 29 Apr 2024 • Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, Guido Zuccon
The current use of large language models (LLMs) for zero-shot document ranking follows one of two ways: 1) prompt-based re-ranking methods, which require no further training but are feasible for only re-ranking a handful of candidate documents due to the associated computational costs; and 2) unsupervised contrastive trained dense retrieval methods, which can retrieve relevant documents from the entire corpus but require a large amount of paired text data for contrastive training.
2 code implementations • 10 Apr 2024 • Ferdinand Schlatt, Maik Fröbe, Harrisen Scells, Shengyao Zhuang, Bevan Koopman, Guido Zuccon, Benno Stein, Martin Potthast, Matthias Hagen
Cross-encoders are effective passage re-rankers.
1 code implementation • 20 Feb 2024 • Shengyao Zhuang, Bevan Koopman, Xiaoran Chu, Guido Zuccon
In this paper, we investigate various aspects of embedding models that could influence the recoverability of text using Vec2Text.
no code implementations • 31 Jan 2024 • Shuai Wang, Shengyao Zhuang, Bevan Koopman, Guido Zuccon
Our ReSLLM method exploits LLMs to drive the selection of resources in federated search in a zero-shot setting.
no code implementations • 24 Jan 2024 • Chuting Yu, Hang Li, Ahmed Mourad, Bevan Koopman, Guido Zuccon
This paper considers Pseudo-Relevance Feedback (PRF) methods for dense retrievers in a resource constrained environment such as that of cheap cloud instances or embedded systems (e. g., smartphones and smartwatches), where memory and CPU are limited and GPUs are not present.
1 code implementation • 16 Jan 2024 • Xinyu Mao, Bevan Koopman, Guido Zuccon
In this context, we show that there is no need for further pre-training if a domain-specific BERT backbone is used within the active learning pipeline.
no code implementations • 12 Jan 2024 • Shuai Wang, Harrisen Scells, Shengyao Zhuang, Martin Potthast, Bevan Koopman, Guido Zuccon
Systematic reviews are crucial for evidence-based medicine as they comprehensively analyse published research findings on specific questions.
no code implementations • 3 Jan 2024 • Shengyao Zhuang, Bevan Koopman, Guido Zuccon
We describe team ielab from CSIRO and The University of Queensland's approach to the 2023 TREC Clinical Trials Track.
1 code implementation • 20 Oct 2023 • Shengyao Zhuang, Bing Liu, Bevan Koopman, Guido Zuccon
In the field of information retrieval, Query Likelihood Models (QLMs) rank documents based on the probability of generating the query given the content of a document.
1 code implementation • 14 Oct 2023 • Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, Guido Zuccon
Our approach reduces the number of LLM inferences and the amount of prompt token consumption during the ranking procedure, significantly improving the efficiency of LLM-based zero-shot ranking.
no code implementations • 17 Sep 2023 • Guido Zuccon, Bevan Koopman, Razia Shaik
We find that ChatGPT provides correct or partially correct answers in about half of the cases (50. 6% of the times), but its suggested references only exist 14% of the times.
1 code implementation • 11 Sep 2023 • Shuai Wang, Harrisen Scells, Martin Potthast, Bevan Koopman, Guido Zuccon
Our best approach is not only viable based on the information available at the time of screening, but also has similar effectiveness to the final title.
1 code implementation • 19 Jul 2023 • Aaron Nicolson, Jason Dowling, Bevan Koopman
To improve diagnostic accuracy, we propose a CXR report generator that integrates aspects of the radiologist workflow and is trained with our proposed reward for reinforcement learning.
no code implementations • 23 Feb 2023 • Guido Zuccon, Bevan Koopman
Aside from measuring the effectiveness of ChatGPT in this context, we show that the knowledge passed in the prompt can overturn the knowledge encoded in the model and this is, in our experiments, to the detriment of answer correctness.
no code implementations • 3 Feb 2023 • Shuai Wang, Harrisen Scells, Bevan Koopman, Guido Zuccon
The ability of ChatGPT to follow complex instructions and generate queries with high precision makes it a valuable tool for researchers conducting systematic reviews, particularly for rapid reviews where time is a constraint and often trading-off higher precision for lower recall is acceptable.
1 code implementation • 21 Dec 2022 • Bevan Koopman, Ahmed Mourad, Hang Li, Anton van der Vegt, Shengyao Zhuang, Simon Gibson, Yash Dang, David Lawrence, Guido Zuccon
On the basis of these needs we release an information retrieval test collection comprising real questions, a large collection of scientific documents split in passages, and ground truth relevance assessments indicating which passages are relevant to each question.
no code implementations • 18 Dec 2022 • Shuai Wang, Harrisen Scells, Bevan Koopman, Guido Zuccon
An empirical analysis compares how effective neural methods compare to traditional methods for this task.
1 code implementation • 19 Sep 2022 • Shuai Wang, Harrisen Scells, Bevan Koopman, Guido Zuccon
However, identifying the correct MeSH terms to include in a query is difficult: information experts are often unfamiliar with the MeSH database and unsure about the appropriateness of MeSH terms for a query.
no code implementations • 12 May 2022 • Hang Li, Ahmed Mourad, Bevan Koopman, Guido Zuccon
Pseudo-Relevance Feedback (PRF) assumes that the top results retrieved by a first-stage ranker are relevant to the original query and uses them to improve the query representation for a second round of retrieval.
1 code implementation • 6 Apr 2022 • Shuai Wang, Harrisen Scells, Justin Clark, Bevan Koopman, Guido Zuccon
However, we show pseudo seed studies are not representative of real seed studies used by information specialists.
1 code implementation • 24 Jan 2022 • Aaron Nicolson, Jason Dowling, Bevan Koopman
Our experimental investigation demonstrates that the Convolutional vision Transformer (CvT) ImageNet-21K and the Distilled Generative Pre-trained Transformer 2 (DistilGPT2) checkpoints are best for warm starting the encoder and decoder, respectively.
no code implementations • 1 Jan 2022 • Duy-Hoa Ngo, Madonna Kemp, Donna Truran, Bevan Koopman, Alejandro Metke-Jimenez
Finding concepts in large clinical ontologies can be challenging when queries use different vocabularies.
1 code implementation • 25 Aug 2021 • Hang Li, Ahmed Mourad, Shengyao Zhuang, Bevan Koopman, Guido Zuccon
Text-based PRF results show that the use of PRF had a mixed effect on deep rerankers across different datasets.