Search Results for author:
Found 105 papers, 59 papers with code
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder
Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space.
基于双星型自注意力网络的搜索结果多样化方法(Search Result Diversification Framework Based on Dual Star-shaped Self-Attention Network)
“相关研究指出, 用户提交给搜索引擎的查询通常为短查询。由于自然语言本身的特点, 短查询通常具有歧义性, 同一个查询可以指代不同的事物, 或同一事物的不同方面。为了让搜索结果尽可能满足用户多样化的信息需求, 搜索引擎需要对返回的结果进行多样化排序, 搜索结果多样化技术应运而生。目前已有的基于全局交互的多样化方法通过全连接的自注意力网络捕获全体候选文档间的交互关系, 取得了较好的效果。但由于此类方法只考虑文档间的相关关系, 并没有考虑到文档是否具有跟查询相关的有效信息, 在训练数据有限的条件下效率相对较低。该文提出了一种基于双星型自注意力网络的搜索结果多样化方法, 将全连接结构改为星型拓扑结构, 并嵌入查询信息以高效率地提取文档跟查询相关的全局交互特征。相关实验结果显示, 该模型相对于基于全连接自注意力网络的多样化方法, 具备显著的性能优势。”
CoRanking: Collaborative Ranking with Small and Large Ranking Agents
Large Language Models (LLMs) have demonstrated superior listwise ranking performance.
HawkBench: Investigating Resilience of RAG Methods on Stratified Information-Seeking Tasks
In real-world information-seeking scenarios, users have dynamic and diverse needs, requiring RAG systems to demonstrate adaptable resilience.
MomentSeeker: A Comprehensive Benchmark and A Strong Baseline For Moment Retrieval Within Long Videos
In this work, we present MomentSeeker, a comprehensive benchmark to evaluate retrieval models' performance in handling general long-video moment retrieval (LVMR) tasks.
mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data
Leveraging these high-quality synthetic and labeled datasets, we train a multimodal multilingual E5 model mmE5.
Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models
To address this issue, we propose a post-training strategy for VGMs, HALO, which explicitly incorporates local feedback from a patch reward model, providing detailed and comprehensive training signals with the video reward model for advanced VGM optimization.
Chain-of-Retrieval Augmented Generation
This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer.
Value Compass Leaderboard: A Platform for Fundamental and Validated Evaluation of LLMs Values
As Large Language Models (LLMs) achieve remarkable breakthroughs, aligning their values with humans has become imperative for their responsible development and customized applications.
Search-o1: Agentic Search-Enhanced Large Reasoning Models
To address this limitation, we introduce \textbf{Search-o1}, a framework that enhances LRMs with an agentic retrieval-augmented generation (RAG) mechanism and a Reason-in-Documents module for refining retrieved documents.
Ranked #1 on
Mathematical Reasoning
on MATH500
Improving GenIR Systems Based on User Feedback
In this chapter, we discuss how to improve the GenIR systems based on user feedback.
A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression
In this work, we provide a thorough investigation of gist-based context compression methods to improve long-context processing in large language models.
Progressive Multimodal Reasoning via Active Retrieval
To bridge the gap in automated multimodal reasoning verification, we employ the MCTS algorithm combined with an active retrieval mechanism, which enables the automatic generation of step-wise annotations.
Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models
Furthermore, we identify two limitations of fine-tuning the full ranking model based on existing methods: (1) sliding window strategy fails to produce a full ranking list as a training label, and (2) the language modeling loss cannot emphasize top-ranked passage IDs in the label.
OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain
As a typical and practical application of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) techniques have gained extensive attention, particularly in vertical domains where LLMs may lack domain-specific knowledge.
Boosting Long-Context Management via Query-Guided Activation Refilling
ACRE constructs a Bi-layer KV Cache for long contexts, where the layer-1 (L1) cache compactly captures global information, and the layer-2 (L2) cache provides detailed and localized information.
RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation
Moreover, to mitigate false pruning in the process of constrained evidence generation, we introduce (1) hierarchical FM-Index constraints, which generate corpus-constrained clues to identify a subset of relevant documents before evidence generation, reducing irrelevant decoding space; and (2) a forward-looking constrained decoding strategy, which considers the relevance of future sequences to improve evidence accuracy.
AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant
The emergence of Large Language Models (LLMs) has significantly advanced natural language processing, but these models often generate factually incorrect information, known as "hallucination".
From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning
The outstanding capabilities of large language models (LLMs) render them a crucial component in various autonomous agent systems.
HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems
To alleviate this problem, we propose HtmlRAG, which uses HTML instead of plain text as the format of retrieved knowledge in RAG.
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation
Retrieval-Augmented Generation (RAG) has become a powerful paradigm for enhancing large language models (LLMs) through external knowledge retrieval.
Little Giants: Synthesizing High-Quality Embedding Data at Scale
In this paper, we introduce SPEED, a framework that aligns open-source small models (8B) to efficiently generate large-scale synthetic embedding data.
A Survey of Conversational Search
As a cornerstone of modern information access, search engines have become indispensable in everyday life.
Toward General Instruction-Following Alignment for Retrieval-Augmented Generation
Due to its robust pipeline design, FollowRAG can seamlessly integrate with different RAG benchmarks.
LLMs + Persona-Plug = Personalized LLMs
By attaching this embedding to the task input, LLMs can better understand and capture user habits and preferences, thereby producing more personalized outputs without tuning their own parameters.
Trustworthiness in Retrieval-Augmented Generation Systems: A Survey
Retrieval-Augmented Generation (RAG) has quickly grown into a pivotal paradigm in the development of Large Language Models (LLMs).
MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge Discovery
Retrieval-Augmented Generation (RAG) leverages retrieval tools to access external databases, thereby enhancing the generation quality of large language models (LLMs) through optimized context.
Towards Effective and Efficient Continual Pre-training of Large Language Models
To make the CPT approach more traceable, this paper presents a technical report for continually pre-training Llama-3 (8B), which significantly enhances the Chinese language ability and scientific reasoning ability of the backbone model.
Query-oriented Data Augmentation for Session Search
However, this paradigm neglects the symmetric nature of the relevance between the session context and document, i. e., the clicked documents can also be paired with different search contexts when training.
Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction
Legal judgment prediction is essential for enhancing judicial efficiency.
YuLan: An Open-source Large Language Model
This paper presents the development of YuLan, a series of open-source LLMs with $12$ billion parameters.
Learning Interpretable Legal Case Retrieval via Knowledge-Guided Case Reformulation
Existing methods in this domain often overlook the incorporation of legal expert knowledge, which is crucial for accurately understanding and modeling legal cases, leading to unsatisfactory retrieval performance.
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
Based on preference data, DPA-RAG accomplishes both external and internal preference alignment: 1) It jointly integrate pair-wise, point-wise, and contrastive preference alignment abilities into the reranker, achieving external preference alignment among RAG components.
Ranked #2 on
Knowledge Base Question Answering
on WebQuestionsSP
DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task
However, few studies have explored how to select appropriate in-context demonstrations for the passage ranking task, which is the focus of this paper.
RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
These ranked documents sufficiently cover various query aspects and are aware of the generator's preferences, hence incentivizing it to produce rich and comprehensive responses for users.
Disentangled Hyperbolic Representation Learning for Heterogeneous Graphs
Therefore, in this paper, we propose $\text{Dis-H}^2\text{GCN}$, a Disentangled Hyperbolic Heterogeneous Graph Convolutional Network.
DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation
We evaluated popular LLMs such as Llama, Baichuan, ChatGLM, and GPT models.
One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models
To address this, we propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
Compressing Lengthy Context With UltraGist
Compressing lengthy context is a critical but technically challenging problem.
Are Long-LLMs A Necessity For Long-Context Tasks?
The learning and deployment of long-LLMs remains a challenging problem despite recent progresses.
FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
Our toolkit and resources are available at https://github. com/RUC-NLPIR/FlashRAG.
Extending Llama-3's Context Ten-Fold Overnight
We extend the context length of Llama-3-8B-Instruct from 8K to 80K via QLoRA fine-tuning.
From Matching to Generation: A Survey on Generative Information Retrieval
Based on the form of information provided to users, current research in GenIR can be categorized into two aspects: \textbf{(1) Generative Document Retrieval} (GR) leverages the generative model's parameters for memorizing documents, enabling retrieval by directly generating relevant document identifiers without explicit indexing.
ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval
Conversational search requires accurate interpretation of user intent from complex multi-turn contexts.
An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
Based on these findings, we then propose several simple document pruning methods to reduce the storage overhead and compare the effectiveness of different pruning methods on different late-interaction models.
UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
To address these challenges, we categorize four available fact sources: human-written evidence, reference documents, search engine results, and LLM knowledge, along with five text generation tasks containing six representative datasets.
Interpreting Conversational Dense Retrieval by Rewriting-Enhanced Inversion of Session Embedding
Such transformation is achieved by training a recently proposed Vec2Text model based on the ad-hoc query encoder, leveraging the fact that the session and query embeddings share the same space in existing conversational dense retrieval.
BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence
Retrieval-augmented large language models (LLMs) have demonstrated efficacy in knowledge-intensive tasks such as open-domain QA, addressing inherent challenges in knowledge update and factual inadequacy.
Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs
The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies.
Metacognitive Retrieval-Augmented Large Language Models
Drawing from cognitive psychology, metacognition allows an entity to self-reflect and critically evaluate its cognitive processes.
Cognitive Personalized Search Integrating Large Language Models with an Efficient Memory Mechanism
To counter this limitation, personalized search has been developed to re-rank results based on user preferences derived from query logs.
Grounding Language Model with Chunking-Free In-Context Retrieval
These strategies not only improve the efficiency of the retrieval process but also ensure that the fidelity of the generated grounding text evidence is maintained.
Enhancing Multi-field B2B Cloud Solution Matching via Contrastive Pre-training
Cloud solutions have gained significant popularity in the technology industry as they offer a combination of services and tools to tackle specific problems.
Generalizing Conversational Dense Retrieval via LLM-Cognition Data Augmentation
ConvAug first generates multi-level augmented conversations to capture the diverse nature of conversational contexts.
CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks
We design the following mechanisms to facilitate effective retrieval and generation, and improve the end-to-end effectiveness of KI tasks: (1) We develop a ranking-oriented DocID list generation strategy, which refines GR by directly learning from a DocID ranking list, to improve retrieval quality.
Session-level Normalization and Click-through Data Enhancement for Session-based Evaluation
Besides, it infers session-level relevance labels based on implicit feedback.
Enhancing Robustness of LLM-Synthetic Text Detectors for Academic Writing: A Comprehensive Analysis
In this paper, we present a comprehensive analysis of the impact of prompts on the text generated by LLMs and highlight the potential lack of robustness in one of the current state-of-the-art GPT detectors.
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning
Despite this, their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language.
Long Context Compression with Activation Beacon
In this paper, we propose Activation Beacon, a plug-in module for transformer-based LLMs that targets effective, efficient, and flexible compression of long contexts.
UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models
Generative information retrieval, encompassing two major tasks of Generative Document Retrieval (GDR) and Grounded Answer Generation (GAR), has gained significant attention in the area of information retrieval and natural language processing.
Retrieve Anything To Augment Large Language Models
On the other hand, the task-specific retrievers lack the required versatility, hindering their performance across the diverse retrieval augmentation scenarios.
Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selection
Previous methods use external knowledge as references for text generation to enhance factuality but often struggle with the knowledge mix-up(e. g., entity mismatch) of irrelevant references.
Large Language Models for Information Retrieval: A Survey
This evolution requires a combination of both traditional methods (such as term-based sparse retrieval methods with rapid response) and modern neural architectures (such as language models with powerful language understanding capacity).
Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community
The research field of Information Retrieval (IR) has evolved significantly, expanding beyond traditional search to meet diverse user information needs.
RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit
To support research in this area and facilitate the development of retrieval-augmented LLM systems, we develop RETA-LLM, a {RET}reival-{A}ugmented LLM toolkit.
User Behavior Simulation with Large Language Model based Agents
Simulating high quality user behavior data has always been a fundamental problem in human-centered applications, where the major difficulty originates from the intricate mechanism of human decision process.
JDsearch: A Personalized Product Search Dataset with Real Queries and Full Interactions
To evaluate the effectiveness of these models, previous studies mainly utilize the simulated Amazon recommendation dataset, which contains automatically generated queries and excludes cold users and tail products.
Generative Retrieval via Term Set Generation
On top of the term-set DocID, we propose a permutation-invariant decoding algorithm, with which the term set can be generated in any permutation yet will always lead to the corresponding document.
WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus
In this paper, we introduce a new NLP task -- generating short factual articles with references for queries by mining supporting evidence from the Web.
Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search
Precisely understanding users' contextual search intent has been an important challenge for conversational search.
CDSM: Cascaded Deep Semantic Matching on Textual Graphs Leveraging Ad-hoc Neighbor Selection
In the first stage, a lightweight CNN-based ad-hod neighbor selector is deployed to filter useful neighbors for the matching task with a small computation cost.
MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling
To tackle these problems, we propose a self-supervised learning framework MCP for capturing better representations from users' dialogue history for personalized chatbots.
Hybrid Inverted Index Is a Robust Accelerator for Dense Retrieval
Based on comprehensive experiments on popular retrieval benchmarks, we verify that clusters and terms indeed complement each other, enabling HI$^2$ to achieve lossless retrieval quality with competitive efficiency across various index settings.
Pre-training for Information Retrieval: Are Hyperlinks Fully Explored?
Hyperlinks, which are commonly used in Web pages, have been leveraged for designing pre-training objectives.
Enhancing User Behavior Sequence Modeling by Generative Tasks for Session Search
To help the encoding of the current user behavior sequence, we propose to use a decoder and the information of future sequences and a supplemental query.
From Easy to Hard: A Dual Curriculum Learning Framework for Context-Aware Document Ranking
In this work, we propose a curriculum learning framework for context-aware document ranking, in which the ranking model learns matching signals between the search context and the candidate document in an easy-to-hard manner.
Ultron: An Ultimate Retriever on Corpus with a Model-based Indexer
In order to unify these two stages, we explore a model-based indexer for document retrieval.
Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding
Our experiments on 13 benchmark datasets across five natural language understanding tasks demonstrate the superiority of our method.
Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation
Existing personalized dialogue systems have tried to extract user profiles from dialogue history to guide personalized response generation.
DynamicRetriever: A Pre-training Model-based IR System with Neither Sparse nor Dense Index
Web search provides a promising way for people to obtain information and has been extensively studied.
KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification and Reasoning Abilities of Language Models
However, to figure out whether PLMs can be reliable knowledge sources and used as alternative knowledge bases (KBs), we need to further explore some critical features of PLMs.
Socialformer: Social Network Inspired Long Document Modeling for Document Ranking
In this paper, we propose the model Socialformer, which introduces the characteristics of social networks into designing sparse attention patterns for long document modeling in document ranking.
PSSL: Self-supervised Learning for Personalized Search with Contrastive Sampling
Personalized search plays a crucial role in improving user search experience owing to its ability to build user profiles based on historical behaviors.
Group based Personalized Search by Integrating Search Behaviour and Friend Network
Specifically, we propose a friend network enhanced personalized search model, which groups the user into multiple friend circles based on search behaviours and friend relations respectively.
Towards More Effective and Economic Sparsely-Activated Model
To increase the number of activated experts without an increase in computational cost, we propose SAM (Switch and Mixture) routing, an efficient hierarchical routing mechanism that activates multiple experts in a same device (GPU).
Learning to Select Historical News Articles for Interaction based Neural News Recommendation
The key to personalized news recommendation is to match the user's interests with the candidate news precisely and efficiently.
USER: A Unified Information Search and Recommendation Model based on Integrated Behavior Sequence
Search and recommendation are the two most common approaches used by people to obtain information.
YES SIR!Optimizing Semantic Space of Negatives with Self-Involvement Ranker
Finally, supervisory signal in rear compressor is computed based on condition probability and thus can control sample dynamic and further enhance the model performance.
Contrastive Learning of User Behavior Sequence for Context-Aware Document Ranking
To learn a more robust representation of the user behavior sequence, we propose a method based on contrastive learning, which takes into account the possible variations in user's behavior sequences.
Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need
In this paper, we propose to leverage the large-scale hyperlinks and anchor texts to pre-train the language model for ad-hoc retrieval.
One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles
Specifically, leveraging the benefits of Transformer on language understanding, we train a personalized language model to construct a general user profile from the user's historical responses.
Learning Implicit User Profiles for Personalized Retrieval-Based Chatbot
To learn a user's personalized language style, we elaborately build language models from shallow to deep using the user's historical responses; To model a user's personalized preferences, we explore the conditional relations underneath each post-response pair of the user.
Proactive Retrieval-based Chatbots based on Relevant Knowledge and Goals
The final response is selected according to the predicted knowledge, the goal to achieve, and the context.
Answer Complex Questions: Path Ranker Is All You Need
Currently, the most popular method for open-domain Question Answering (QA) adopts "Retriever and Reader" pipeline, where the retriever extracts a list of candidate documents from a large set of documents followed by a ranker to rank the most relevant documents and the reader extracts answer from the candidates.
Emotion Eliciting Machine: Emotion Eliciting Conversation Generation based on Dual Generator
Then we propose a dual encoder-decoder structure to model the generation of responses in both positive and negative side based on the changes of the user's emotion status in the conversation.
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
We further construct a large Chinese multi-source image-text dataset called RUC-CAS-WenLan for pre-training our BriVL model.
Ranked #1 on
Image Retrieval
on RUC-CAS-WenLan
Less is More: Pre-train a Strong Text Encoder for Dense Retrieval Using a Weak Decoder
Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space.
Neural Sentence Ordering Based on Constraint Graphs
Our experiments on five benchmark datasets show that our method outperforms all the existing baselines significantly, achieving a new state-of-the-art performance.
Content Selection Network for Document-grounded Retrieval-based Chatbots
It is thus crucial to select the part of document content relevant to the current conversation context.
Pchatbot: A Large-Scale Dataset for Personalized Chatbot
This enables the development of personalized dialogue models that directly learn implicit user personality from the user's dialogue history.
ScriptWriter: Narrative-Guided Script Generation
In dialogue systems, it would also be useful to drive dialogues by a dialogue plan.
Personalizing Search Results Using Hierarchical RNN with Query-aware Attention
Our analysis reveals that the attention model is able to attribute higher weights to more related past sessions after fine training.
