Search Results for author:
Found 22 papers, 15 papers with code
Does RAG Really Perform Bad For Long-Context Processing?
The efficient processing of long context poses a serious challenge for large language models (LLMs).
Search-o1: Agentic Search-Enhanced Large Reasoning Models
To address this limitation, we introduce \textbf{Search-o1}, a framework that enhances LRMs with an agentic retrieval-augmented generation (RAG) mechanism and a Reason-in-Documents module for refining retrieved documents.
Ranked #1 on
Mathematical Reasoning
on MATH500
Boosting Long-Context Management via Query-Guided Activation Refilling
ACRE constructs a Bi-layer KV Cache for long contexts, where the layer-1 (L1) cache compactly captures global information, and the layer-2 (L2) cache provides detailed and localized information.
Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding
The VST module is trained by instruction fine-tuning, where two optimizing strategies are offered.
MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge Discovery
Retrieval-Augmented Generation (RAG) leverages retrieval tools to access external databases, thereby enhancing the generation quality of large language models (LLMs) through optimized context.
Compressing Lengthy Context With UltraGist
Compressing lengthy context is a critical but technically challenging problem.
Are Long-LLMs A Necessity For Long-Context Tasks?
The learning and deployment of long-LLMs remains a challenging problem despite recent progresses.
Extending Llama-3's Context Ten-Fold Overnight
We extend the context length of Llama-3-8B-Instruct from 8K to 80K via QLoRA fine-tuning.
From Matching to Generation: A Survey on Generative Information Retrieval
We will summarize the advancements in GR regarding model training, document identifier, incremental learning, downstream tasks adaptation, multi-modal GR and generative recommendation, as well as progress in reliable response generation in aspects of internal knowledge memorization, external knowledge augmentation, generating response with citations and personal information assistant.
Extensible Embedding: A Flexible Multipler For LLM's Context Length
2) Strong sample efficiency of training, which enables the embedding model to be learned in a cost-effective way.
BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
It can simultaneously perform the three common retrieval functionalities of embedding model: dense retrieval, multi-vector retrieval, and sparse retrieval, which provides a unified model foundation for real-world IR applications.
Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization
Extensible Tokenization stands as a midware in between of the tokenized context and the LLM, which transforms the raw token embeddings into the extensible embeddings.
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning
Despite this, their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language.
Long Context Compression with Activation Beacon
In this paper, we propose Activation Beacon, a plug-in module for transformer-based LLMs that targets effective, efficient, and flexible compression of long contexts.
LM-Cocktail: Resilient Tuning of Language Models via Model Merging
Despite simplicity, LM-Cocktail is surprisingly effective: the resulted model is able to achieve a strong empirical performance in the whole scope of general tasks while preserving a superior capacity in its targeted domain.
Retrieve Anything To Augment Large Language Models
On the other hand, the task-specific retrievers lack the required versatility, hindering their performance across the diverse retrieval augmentation scenarios.
C-Pack: Packed Resources For General Chinese Embeddings
Along with our resources on general Chinese embedding, we release our data and models for English text embeddings.
Generative Retrieval via Term Set Generation
On top of the term-set DocID, we propose a permutation-invariant decoding algorithm, with which the term set can be generated in any permutation yet will always lead to the corresponding document.
Hybrid Inverted Index Is a Robust Accelerator for Dense Retrieval
Based on comprehensive experiments on popular retrieval benchmarks, we verify that clusters and terms indeed complement each other, enabling HI$^2$ to achieve lossless retrieval quality with competitive efficiency across various index settings.
Ultron: An Ultimate Retriever on Corpus with a Model-based Indexer
In order to unify these two stages, we explore a model-based indexer for document retrieval.
GateFormer: Speeding Up News Feed Recommendation with Input Gated Transformers
News feed recommendation is an important web service.
Learning to Select Historical News Articles for Interaction based Neural News Recommendation
The key to personalized news recommendation is to match the user's interests with the candidate news precisely and efficiently.
