Search Results for author:
Found 73 papers, 43 papers with code
Community-level Research on Suicidality Prediction in a Secure Environment: Overview of the CLPsych 2021 Shared Task
Progress on NLP for mental health — indeed, for healthcare in general — is hampered by obstacles to shared, community-level access to relevant data.
TBD3: A Thresholding-Based Dynamic Depression Detection from Social Media for Low-Resource Users
To complement this evaluation, we propose a dynamic thresholding technique that adjusts the classifier’s sensitivity as a function of the number of posts a user has.
An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc
Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists.
Lost in Transliteration: Bridging the Script Gap in Neural IR
This creates a ``script gap" between the performance of the same queries when written in their native or transliterated form.
Artifact Sharing for Information Retrieval Research
Sharing artifacts -- such as trained models, pre-built indexes, and the code to use them -- aids in reproducibility efforts by allowing researchers to validate intermediate steps and improves the sustainability of research by allowing multiple groups to build off one another's prior computational work.
Document Quality Scoring for Web Crawling
The internet contains large amounts of low-quality content, yet users expect web search engines to deliver high-quality, relevant results.
MURR: Model Updating with Regularized Replay for Searching a Document Stream
The Internet produces a continuous stream of new documents and user-generated queries.
On Precomputation and Caching in Information Retrieval Experiments with Pipeline Architectures
These approaches allow for the best of both worlds: pipelines can be fully expressed end-to-end, while also avoiding redundant computations between pipelines.
Breaking the Lens of the Telescope: Online Relevance Estimation over Large Retrieval Sets
Instead of re-ranking a fixed set of top-k documents in a single step, online relevance estimation iteratively re-scores smaller subsets of the most promising documents while adjusting relevance scores for the remaining pool based on the estimations from the final model using an online bandit-based algorithm.
Efficient Constant-Space Multi-Vector Retrieval
Multi-vector retrieval methods, exemplified by the ColBERT architecture, have shown substantial promise for retrieval by providing strong trade-offs in terms of retrieval latency and effectiveness.
Improving Low-Resource Retrieval Effectiveness using Zero-Shot Linguistic Similarity Transfer
We find that this approach improves the performance of the varieties upon which the models were directly trained, thereby regularising these models to generalise and perform better even on unseen language variety pairs.
Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers
State-of-the-art cross-encoders can be fine-tuned to be highly effective in passage re-ranking.
Variations in Relevance Judgments and the Shelf Life of Test Collections
The fundamental property of Cranfield-style evaluations, that system rankings are stable even when assessors disagree on individual relevance decisions, was validated on traditional test collections.
mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval
We see strong cross-lingual performance with English-based retrievers that trained using instructions, but find a notable drop in performance in the multilingual setting, indicating that more work is needed in developing data for instruction-based multilingual retrievers.
MechIR: A Mechanistic Interpretability Framework for Information Retrieval
Mechanistic interpretability is an emerging diagnostic approach for neural models that has gained traction in broader natural language processing domains.
Guiding Retrieval using LLM-based Listwise Rankers
In this paper, we propose an adaptation of an existing adaptive retrieval method that supports the listwise setting and helps guide the retrieval process itself (thereby overcoming the bounded recall problem for LLM rerankers).
Training on the Test Model: Contamination in Ranking Distillation
By simulating a ``worst-case'' setting where the degree of contamination is known, we find that contamination occurs even when the test data represents a small fraction of the teacher's training samples.
Quam: Adaptive Retrieval through Query Affinity Modelling
Our extensive experimental evidence shows that our proposed approach, Quam improves the recall performance by up to 26\% over the standard re-ranking baselines.
DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained transformers, which often split entities into nonsensical fragments.
Genetic Approach to Mitigate Hallucination in Generative IR
We address hallucination by adapting an existing genetic generation approach with a new 'balanced fitness function' consisting of a cross-encoder model for relevance and an n-gram overlap metric to promote grounding.
LexBoost: Improving Lexical Document Retrieval with Nearest Neighbors
For efficiency, approximation methods like HNSW are frequently used to approximate exhaustive dense retrieval.
Neural Passage Quality Estimation for Static Pruning
We refer to this query-agnostic estimation of passage relevance as a passage's quality.
Top-Down Partitioning for Efficient List-Wise Ranking
Large Language Models (LLMs) have significantly impacted many facets of natural language processing and information retrieval.
Generative Relevance Feedback and Convergence of Adaptive Re-Ranking: University of Glasgow Terrier Team at TREC DL 2023
In re-ranking, we investigate operating points of adaptive re-ranking with different first stages to find the point in graph traversal where the first stage no longer has an effect on the performance of the overall retrieval pipeline.
On the Evaluation of Machine-Generated Reports
Reports with these qualities are necessary to satisfy the complex, nuanced, or multi-faceted information needs of users.
Exploiting Positional Bias for Query-Agnostic Generative Content in Search
We demonstrate such defects by showing that non-relevant text--such as promotional content--can be easily injected into a document without adversely affecting its position in search results.
A Reproducibility Study of PLAID
The PLAID (Performance-optimized Late Interaction Driver) algorithm for ColBERTv2 uses clustered term representations to retrieve and progressively prune documents for final (exact) document scoring.
Overview of the TREC 2023 NeuCLIR Track
The principal tasks are ranked retrieval of news in one of the three languages, using English topics.
Shallow Cross-Encoders for Low-Latency Retrieval
However, Cross-Encoders based on large transformer models (such as BERT or T5) are computationally expensive and allow for scoring only a small number of documents within a reasonably small latency window.
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
First, we introduce our dataset FollowIR, which contains a rigorous instruction evaluation benchmark as well as a training set for helping IR models learn to better follow real-world instructions.
Analyzing Adversarial Attacks on Sequence-to-Sequence Relevance Models
Modern sequence-to-sequence relevance models like monoT5 can effectively capture complex textual interactions between queries and documents through cross-encoding.
Evaluating the Explainability of Neural Rankers
While the increasing complexity of the search models have been able to demonstrate improvements in effectiveness (measured in terms of relevance of top-retrieved results), a question worthy of a thorough inspection is - "how explainable are these models?
A Deep Learning Approach for Selective Relevance Feedback
Additionally, to further improve retrieval effectiveness with this selective PRF approach, we make use of the model's confidence estimates to combine the information from the original and expanded queries.
Generative Query Reformulation for Effective Adhoc Search
GenQR directly reformulates the user's input query, while GenPRF provides additional context for the query by making use of pseudo-relevance feedback information.
On the Effects of Regional Spelling Conventions in Retrieval Models
One advantage of neural ranking models is that they are meant to generalise well in situations of synonymity i. e. where two words have similar or identical meanings.
Lexically-Accelerated Dense Retrieval
We introduce 'LADR' (Lexically-Accelerated Dense Retrieval), a simple-yet-effective approach that improves the efficiency of existing dense retrieval models without compromising on retrieval effectiveness.
Adaptive Latent Entity Expansion for Document Retrieval
First, we demonstrate that applying a strong neural re-ranker before sparse or dense PRF can improve the retrieval effectiveness by 5-8%.
Online Distillation for Pseudo-Relevance Feedback
Model distillation has emerged as a prominent technique to improve neural search models.
The Information Retrieval Experiment Platform
Standardization is achieved when a retrieval approach implements PyTerrier's interfaces and the input and output of an experiment are compatible with ir_datasets and ir_measures.
Adapting Learned Sparse Retrieval for Long Documents
We investigate existing aggregation approaches for adapting LSR to longer documents and find that proximal scoring is crucial for LSR to handle long documents.
Overview of the TREC 2022 NeuCLIR Track
This is the first year of the TREC Neural CLIR (NeuCLIR) track, which aims to study the impact of neural approaches to cross-language information retrieval.
A Unified Framework for Learned Sparse Retrieval
We then reproduce all prominent methods using a common codebase and re-train them in the same environment, which allows us to quantify how components of the framework affect effectiveness and efficiency.
One-Shot Labeling for Automatic Relevance Estimation
We then explore various approaches for predicting the relevance of unjudged documents with respect to a query and the known relevant document, including nearest neighbor, supervised, and prompting techniques.
Doc2Query--: When Less is More
Doc2Query -- the process of expanding the content of a document before indexing using a sequence-to-sequence model -- has emerged as a prominent technique for improving the first-stage retrieval effectiveness of search engines.
Adaptive Re-Ranking with a Corpus Graph
Search systems often employ a re-ranking pipeline, wherein documents (or passages) from an initial pool of candidates are assigned new ranking scores.
CODEC: Complex Document and Entity Collection
We also show that the manual query reformulations significantly improve document ranking and entity ranking performance.
On Survivorship Bias in MS MARCO
We observe that this bias could be present in the popular MS MARCO dataset, given that annotators could not find answers to 38--45% of the queries, leading to these queries being discarded in training and evaluation processes.
Reproducing Personalised Session Search over the AOL Query Log
Given that web documents are prone to change over time, we study the differences present between a version of the corpus containing documents as they appeared in 2017 (which has been used by several recent works) and a new version we construct that includes documents close to as they appeared at the time the query log was produced (2006).
Streamlining Evaluation with ir-measures
We present ir-measures, a new tool that makes it convenient to calculate a diverse set of evaluation measures used in information retrieval.
Max-Utility Based Arm Selection Strategy For Sequential Query Recommendations
We show that such a selection strategy often results in higher cumulative regret and to this end, we propose a selection strategy based on the maximum utility of the arms.
IntenT5: Search Result Diversification using Causal Language Models
Existing approaches often rely on massive query logs and interaction data to generate a variety of possible query intents, which then can be used to re-rank documents.
Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Review
We indeed find that the pre-trained BERT model reduces review cost by 10% to 15% in TAR workflows simulated on the RCV1-v2 newswire collection.
Simplified Data Wrangling with ir_datasets
Managing the data for Information Retrieval (IR) experiments can be challenging.
ToxCCIn: Toxic Content Classification with Interpretability
Despite the recent successes of transformer-based models in terms of effectiveness on a variety of tasks, their decisions often remain opaque to humans.
ABNIRML: Analyzing the Behavior of Neural IR Models
Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc search.
SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search
With worldwide concerns surrounding the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), there is a rapidly growing body of scientific literature on the virus.
PARADE: Passage Representation Aggregation for Document Reranking
In this work, we explore strategies for aggregating relevance signals from a document's passages into a final ranking score.
Ranked #2 on
Ad-Hoc Information Retrieval
on TREC Robust04
GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection
Offensive language detection is an important and challenging task in natural language processing.
Interaction Matching for Long-Tail Multi-Label Classification
We present an elegant and effective approach for addressing limitations in existing multi-label classification models by incorporating interaction matching, a concept shown to be useful for ad-hoc search result ranking.
SLEDGE: A Simple Yet Effective Baseline for COVID-19 Scientific Knowledge Search
In this work, we present a search system called SLEDGE, which utilizes SciBERT to effectively re-rank articles.
Training Curricula for Open Domain Answer Re-Ranking
We show that the proposed heuristics can be used to build a training curriculum that down-weights difficult samples early in the training process.
Expansion via Prediction of Importance with Contextualization
We also observe that the performance is additive with the current leading first-stage retrieval methods, further narrowing the gap between inexpensive and cost-prohibitive passage ranking approaches.
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations
Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking.
Ranking Significant Discrepancies in Clinical Reports
This allows medical practitioners to easily identify and learn from the reports in which their interpretation most substantially differed from that of the attending physician (who finalized the report).
Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning
While billions of non-English speaking users rely on search engines every day, the problem of ad-hoc information retrieval is rarely studied for non-English languages.
Ontology-Aware Clinical Abstractive Summarization
Automatically generating accurate summaries from clinical reports could save a clinician's time, improve summary coverage, and reduce errors.
CEDR: Contextualized Embeddings for Document Ranking
We call this joint approach CEDR (Contextualized Embeddings for Document Ranking).
Ranked #3 on
Ad-Hoc Information Retrieval
on TREC Robust04
RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses
Self-reported diagnosis statements have been widely employed in studying language related to mental health in social media.
SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions
Mental health is a significant and growing public health concern.
A Deeper Look into Dependency-Based Word Embeddings
We investigate the effect of various dependency-based word embeddings on distinguishing between functional and domain similarity, word similarity rankings, and two downstream tasks in English.
GU IRLAB at SemEval-2018 Task 7: Tree-LSTMs for Scientific Relation Classification
SemEval 2018 Task 7 focuses on relation ex- traction and classification in scientific literature.
GUIR at SemEval-2017 Task 12: A Framework for Cross-Domain Clinical Temporal Information Extraction
Clinical TempEval 2017 (SemEval 2017 Task 12) addresses the task of cross-domain temporal extraction from clinical text.
Content-Based Weak Supervision for Ad-Hoc Re-Ranking
One challenge with neural ranking is the need for a large amount of manually-labeled relevance judgments for training.
