Search Results for author:
Found 22 papers, 18 papers with code
Matching-oriented Embedding Quantization For Ad-hoc Retrieval
In this work, we propose the Matching-oriented Product Quantization (MoPQ), where a novel objective Multinoulli Contrastive Loss (MCL) is formulated.
BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models
In this work, we proposeExtensible Embedding, which realizes high-quality extension of LLM's context with strong flexibility and cost-effectiveness.
Extensible Embedding: A Flexible Multipler For LLM's Context Length
2) Strong sample efficiency of training, which enables the embedding model to be learned in a cost-effective way.
BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
It can simultaneously perform the three common retrieval functionalities of embedding model: dense retrieval, multi-vector retrieval, and sparse retrieval, which provides a unified model foundation for real-world IR applications.
Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization
Extensible Tokenization stands as a midware in between of the tokenized context and the LLM, which transforms the raw token embeddings into the extensible embeddings.
Soaring from 4K to 400K: Extending LLM's Context with Activation Beacon
Although the context window can be extended through fine-tuning, it will result in a considerable cost at both training and inference time, and exert an unfavorable impact to the LLM's original capabilities.
Making Large Language Models A Better Foundation For Dense Retrieval
LLaRA consists of two pretext tasks: EBAE (Embedding-Based Auto-Encoding) and EBAR (Embedding-Based Auto-Regression), where the text embeddings from LLM are used to reconstruct the tokens for the input sentence and predict the tokens for the next sentence, respectively.
LM-Cocktail: Resilient Tuning of Language Models via Model Merging
Despite simplicity, LM-Cocktail is surprisingly effective: the resulted model is able to achieve a strong empirical performance in the whole scope of general tasks while preserving a superior capacity in its targeted domain.
Retrieve Anything To Augment Large Language Models
On the other hand, the task-specific retrievers lack the required versatility, hindering their performance across the diverse retrieval augmentation scenarios.
C-Pack: Packaged Resources To Advance General Chinese Embedding
Along with our resources on general Chinese embedding, we release our data and models for English text embeddings.
RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
It is designed to improve the quality of semantic representation where all contextualized embeddings of the pre-trained model can be leveraged.
RetroMAE v2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
DupMAE, which targets on improving the semantic representation capacity for the contextualized embeddings of both [CLS] and ordinary tokens.
Ranked #1 on
Information Retrieval
on MS MARCO
(MRR@10 metric)
Hybrid Inverted Index Is a Robust Accelerator for Dense Retrieval
Based on comprehensive experiments on popular retrieval benchmarks, we verify that clusters and terms indeed complement each other, enabling HI$^2$ to achieve lossless retrieval quality with competitive efficiency across various index settings.
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder
The sentence embedding is generated from the encoder's masked input; then, the original sentence is recovered based on the sentence embedding and the decoder's masked input via masked language modeling.
Ranked #1 on
Information Retrieval
on MSMARCO
Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings
We perform comprehensive explorations for the optimal conduct of knowledge distillation, which may provide useful insights for the learning of VQ based ANN index.
A Mutually Reinforced Framework for Pretrained Sentence Embeddings
Instead of relying on annotation heuristics defined by humans, it leverages the sentence representation model itself and realizes the following iterative self-supervision process: on one hand, the improvement of sentence representation may contribute to the quality of data annotation; on the other hand, more effective data annotation helps to generate high-quality positive samples, which will further improve the current sentence representation model.
Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Search
On the other hand, the capability of making high-CTR retrieval is optimized by learning to discriminate user's clicked ads from the entire corpus.
Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval
In this work, we tackle this problem with Bi-Granular Document Representation, where the lightweight sparse embeddings are indexed and standby in memory for coarse-grained candidate search, and the heavyweight dense embeddings are hosted in disk for fine-grained post verification.
LECF: Recommendation via Learnable Edge Collaborative Filtering
In this paper, we model an interaction between user and item as an edge and propose a novel CF framework, called learnable edge collaborative filtering (LECF).
GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph
The representation learning on textual graph is to generate low-dimensional embeddings for the nodes based on the individual textual features and the neighbourhood information.
Matching-oriented Product Quantization For Ad-hoc Retrieval
In this work, we propose the Matching-oriented Product Quantization (MoPQ), where a novel objective Multinoulli Contrastive Loss (MCL) is formulated.
Training Large-Scale News Recommenders with Pretrained Language Models in the Loop
Secondly, it improves the data efficiency of the training workflow, where non-informative data can be eliminated from encoding.
