Search Results for author:
Found 22 papers, 6 papers with code
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning
Research on neural IR has so far been focused primarily on standard supervised learning settings, where it outperforms traditional term matching baselines.
Self-Refinement of Language Models from External Proxy Metrics Feedback
It is often desirable for Large Language Models (LLMs) to capture multiple objectives when providing a response.
Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations
We introduce a structured chain-of-thought (SCoT) prompting approach to generating content-grounded multi-turn question-answer conversations using a pre-trained large language model (LLM).
An Empirical Investigation into the Effect of Parameter Choices in Knowledge Distillation
We present a large-scale empirical study of how choices of configuration parameters affect performance in knowledge distillation (KD).
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation
In this paper, we present the discovery that a student model distilled from a few-shot prompted LLM can commonly generalize better than its teacher to unseen examples on such tasks.
Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs
Using in-context learning (ICL) for data generation, techniques such as Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023) can train strong conversational agents with only a small amount of human supervision.
Inference-time Re-ranker Relevance Feedback for Neural Information Retrieval
Neural information retrieval often adopts a retrieve-and-rerank framework: a bi-encoder network first retrieves K (e. g., 100) candidates that are then re-ranked using a more powerful cross-encoder model to rank the better candidates higher.
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers
Many information retrieval tasks require large labeled datasets for fine-tuning.
Knowledge Distillation $\approx$ Label Smoothing: Fact or Fallacy?
Originally proposed as a method for knowledge transfer from one model to another, some recent studies have suggested that knowledge distillation (KD) is in fact a form of regularization.
PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development
The field of Question Answering (QA) has made remarkable progress in recent years, thanks to the advent of large pre-trained language models, newer realistic benchmark datasets with leaderboards, and novel algorithms for key components such as retrievers and readers.
Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking
Neural information retrieval (IR) systems have progressed rapidly in recent years, in large part due to the release of publicly available benchmarking tasks.
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers
Fine-tuning pre-trained language models (PLMs) achieves impressive performance on a range of downstream tasks, and their sizes have consequently been getting bigger.
GAAMA 2.0: An Integrated System that Answers Boolean and Extractive Questions
Recent machine reading comprehension datasets include extractive and boolean questions but current approaches do not offer integrated support for answering both question types.
Not to Overfit or Underfit the Source Domains? An Empirical Study of Domain Generalization in Question Answering
Machine learning models are prone to overfitting their training (source) domains, which is commonly believed to be the reason why they falter in novel target domains.
Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval
On two public IR benchmarks, we empirically show that the proposed method helps improve both the model's attention patterns and retrieval performance, including in zero-shot settings.
Synthetic Target Domain Supervision for Open Retrieval QA
Neural passage retrieval is a new and promising approach in open retrieval question answering.
Learning Cross-Lingual IR from an English Retriever
We present DR. DECR (Dense Retrieval with Distillation-Enhanced Cross-Lingual Representation), a new cross-lingual information retrieval (CLIR) system trained using multi-stage knowledge distillation (KD).
Cross-Lingual Information Retrieval
Knowledge Distillation
+3
Towards Robust Neural Retrieval Models with Synthetic Pre-Training
Recent work has shown that commonly available machine reading comprehension (MRC) datasets can be used to train high-performance neural information retrieval (IR) systems.
End-to-End QA on COVID-19: Domain Adaptation with Synthetic Training
End-to-end question answering (QA) requires both information retrieval (IR) over a large document collection and machine reading comprehension (MRC) on the retrieved passages.
Answer Span Correction in Machine Reading Comprehension
Answer validation in machine reading comprehension (MRC) consists of verifying an extracted answer against an input context and question pair.
Improved Synthetic Training for Reading Comprehension
Automatically generated synthetic training examples have been shown to improve performance in machine reading comprehension (MRC).
Multi-Stage Pre-training for Low-Resource Domain Adaptation
Transfer learning techniques are particularly useful in NLP tasks where a sizable amount of high-quality annotated data is difficult to obtain.
