Search Results for author:
Found 18 papers, 10 papers with code
Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs?
While metrics available during pre-training, such as perplexity, correlate well with model performance at scaling-laws studies, their predictive capacities at a fixed model size remain unclear, hindering effective model selection and development.
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
Efficiently acquiring external knowledge and up-to-date information is essential for effective reasoning and text generation in large language models (LLMs).
ATEB: Evaluating and Improving Advanced NLP Tasks for Text Embedding Models
We introduce a new benchmark designed to assess and highlight the limitations of embedding models trained on existing information retrieval data mixtures on advanced capabilities, which include factuality, safety, instruction following, reasoning and document-level understanding.
Scaling Sparse and Dense Retrieval in Decoder-Only LLMs
Scaling large language models (LLMs) has shown great potential for improving retrieval model performance; however, previous studies have mainly focused on dense retrieval trained with contrastive loss (CL), neglecting the scaling behavior of other retrieval paradigms and optimization techniques, such as sparse retrieval and knowledge distillation (KD).
Hypencoder: Hypernetworks for Information Retrieval
To produce the small neural network we use a hypernetwork, a network that produces the weights of other networks, as our query encoder.
Inference Scaling for Long-Context Retrieval Augmented Generation
Our observations reveal that increasing inference computation leads to nearly linear gains in RAG performance when optimally allocated, a relationship we describe as the inference scaling laws for RAG.
Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Models
In our experiments, using graph-guided response simulations leads to significant improvements in intent classification, slot filling and response relevance compared to naive single-prompt simulated conversations.
Conversational Question Answering
Dialogue State Tracking
+8
Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding
This paper introduces PAG-a novel optimization and decoding approach that guides autoregressive generation of document identifiers in generative retrieval models through simultaneous decoding.
Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond
Developing a universal model that can effectively harness heterogeneous resources and respond to a wide range of personalized needs has been a longstanding community aspiration.
Scalable and Effective Generative Information Retrieval
This paper represents an important milestone in generative retrieval research by showing, for the first time, that generative retrieval models can be trained to perform effectively on large-scale standard retrieval benchmarks.
Topic Segmentation of Semi-Structured and Unstructured Conversational Datasets using Language Models
However, current works on topic segmentation often focus on segmentation of structured texts.
Language Models As Semantic Indexers
Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs.
Soft Prompt Decoding for Multilingual Dense Retrieval
In this work, we explore a Multilingual Information Retrieval (MLIR) task, where the collection includes documents in multiple languages.
Cross-Lingual Information Retrieval
Knowledge Distillation
+1
A Personalized Dense Retrieval Framework for Unified Information Access
Developing a universal model that can efficiently and effectively respond to a wide range of information access requests -- from retrieval to recommendation to question answering -- has been a long-lasting goal in the information retrieval community.
Curriculum Learning for Dense Retrieval Distillation
Recent work has shown that more effective dense retrieval models can be obtained by distilling ranking knowledge from an existing base re-ranking model.
Understanding the Effectiveness of Reviews in E-commerce Top-N Recommendation
We find that models utilizing only review information can not achieve better performances than vanilla implicit-feedback matrix factorization method.
A Zero Attentive Relevance Matching Networkfor Review Modeling in Recommendation System
User and item reviews are valuable for the construction of recommender systems.
A Hierarchical Self-attentive Convolution Network for Review Modeling in Recommendation Systems
Using reviews to learn user and item representations is important for recommender system.
