Search Results for author:
Found 42 papers, 16 papers with code
Matching-oriented Embedding Quantization For Ad-hoc Retrieval
In this work, we propose the Matching-oriented Product Quantization (MoPQ), where a novel objective Multinoulli Contrastive Loss (MCL) is formulated.
Don't Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript
Recent years have witnessed intensive research interests on training deep neural networks (DNNs) more efficiently by quantization-based compression methods, which facilitate DNNs training in two ways: (1) activations are quantized to shrink the memory consumption, and (2) gradients are quantized to decrease the communication cost.
Token-Efficient Leverage Learning in Large Language Models
With the same amount of task data, TELL leads in improving task performance compared to SFT.
LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model
Furthermore, we utilize the LLM to enrich the information of personality labels for enhancing the detection performance.
Making Large Language Models A Better Foundation For Dense Retrieval
LLaRA consists of two pretext tasks: EBAE (Embedding-Based Auto-Encoding) and EBAR (Embedding-Based Auto-Regression), where the text embeddings from LLM are used to reconstruct the tokens for the input sentence and predict the tokens for the next sentence, respectively.
Experimental Analysis of Large-scale Learnable Vector Storage Compression
Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains.
Relation Extraction Model Based on Semantic Enhancement Mechanism
The CasAug model proposed in this paper based on the CasRel framework combined with the semantic enhancement mechanism can solve this problem to a certain extent.
Dynamic Fair Federated Learning Based on Reinforcement Learning
Federated learning enables a collaborative training and optimization of global models among a group of devices without sharing local data samples.
Entity Alignment Method of Science and Technology Patent based on Graph Convolution Network and Information Fusion
The entity alignment of science and technology patents aims to link the equivalent entities in the knowledge graph of different science and technology patent data sources.
Accelerating Scalable Graph Neural Network Inference with Node-Adaptive Propagation
To further accelerate Scalable GNNs inference in this inductive setting, we propose an online propagation framework and two novel node-adaptive propagation methods that can customize the optimal propagation depth for each node based on its topological information and thereby avoid redundant feature propagation.
Reinforcement Federated Learning Method Based on Adaptive OPTICS Clustering
The algorithm combines OPTICS clustering and adaptive learning technology, and can effective-ly deal with the problem of non-independent and identically distributed data across different user terminals.
RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
It is designed to improve the quality of semantic representation where all contextualized embeddings of the pre-trained model can be leveraged.
Distributed Graph Neural Network Training: A Survey
In recent years, many efforts have been made on distributed GNN training, and an array of training algorithms and systems have been proposed.
Efficient Graph Neural Network Inference at Large Scale
Graph neural networks (GNNs) have demonstrated excellent performance in a wide range of applications.
Adaptive Dual Channel Convolution Hypergraph Representation Learning for Technological Intellectual Property
Therefore, a hypergraph neural network model based on dual channel convolution is proposed.
A Relational Triple Extraction Method Based on Feature Reasoning for Technological Patents
However, most of them only establish separate table features for each relationship, which ignores the implicit relationship between different entity pairs and different relationship features.
Diffusion Models: A Comprehensive Survey of Methods and Applications
This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration.
A Rare Topic Discovery Model for Short Texts Based on Co-occurrence word Network
We provide a simple and general solution for the discovery of scarce topics in unbalanced short-text datasets, namely, a word co-occurrence network-based model CWIBTD, which can simultaneously address the sparsity and unbalance of short-text topics and attenuate the effect of occasional pairwise occurrences of words, allowing the model to focus more on the discovery of scarce topics.
A sentiment analysis model for car review texts based on adversarial training and whole word mask BERT
Based on this, we propose a car review text sentiment analysis model based on adversarial training and whole word mask BERT(ATWWM-BERT).
Sentiment Analysis of Online Travel Reviews Based on Capsule Network and Sentiment Lexicon
In this paper, we study the text sentiment classification of online travel reviews based on social media online comments and propose the SCCL model based on capsule network and sentiment lexicon.
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder
The sentence embedding is generated from the encoder's masked input; then, the original sentence is recovered based on the sentence embedding and the decoder's masked input via masked language modeling.
Ranked #1 on
Information Retrieval
on MSMARCO
Profiling and Evolution of Intellectual Property
In recent years, with the rapid growth of Internet data, the number and types of scientific and technological resources are also rapidly expanding.
Retrieval of Scientific and Technological Resources for Experts and Scholars
The scientific and technological resources of experts and scholars are mainly composed of basic attributes and scientific research achievements.
Research on Intellectual Property Resource Profile and Evolution Law
In the era of big data, intellectual property-oriented scientific and technological resources show the trend of large data scale, high information density and low value density, which brings severe challenges to the effective use of intellectual property resources, and the demand for mining hidden information in intellectual property is increasing.
Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings
We perform comprehensive explorations for the optimal conduct of knowledge distillation, which may provide useful insights for the learning of VQ based ANN index.
Scientific and Technological Text Knowledge Extraction Method of based on Word Mixing and GRU
The knowledge extraction task is to extract triple relations (head entity-relation-tail entity) from unstructured text data.
An Intellectual Property Entity Recognition Method Based on Transformer and Technological Word Information
This paper proposes a method for extracting intellectual property entities based on Transformer and technical word information , and provides accurate word vector representation in combination with the BERT language method.
Web Page Content Extraction Based on Multi-feature Fusion
With the rapid growth of the number and types of web resources, there are still problems to be solved when using a single strategy to extract the text information of different pages.
Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network
Heterogeneous Graph Neural Network (HGNN) has been successfully employed in various tasks, but we cannot accurately know the importance of different design dimensions of HGNNs due to diverse architectures and applied scenarios.
Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Search
On the other hand, the capability of making high-CTR retrieval is optimized by learning to discriminate user's clicked ads from the entire corpus.
Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval
In this work, we tackle this problem with Bi-Granular Document Representation, where the lightweight sparse embeddings are indexed and standby in memory for coarse-grained candidate search, and the heavyweight dense embeddings are hosted in disk for fine-grained post verification.
LECF: Recommendation via Learnable Edge Collaborative Filtering
In this paper, we model an interaction between user and item as an edge and propose a novel CF framework, called learnable edge collaborative filtering (LECF).
Self-Supervised Graph Co-Training for Session-based Recommendation
In this paper, for informative session-based data augmentation, we combine self-supervised learning with co-training, and then develop a framework to enhance session-based recommendation.
Memory-aware framework for fast and scalable second-order random walk over billion-edge natural graphs
In this paper, to clearly compare the efficiency of various node sampling methods, we first design a cost model and propose two new node sampling methods: one follows the acceptance-rejection paradigm to achieve a better balance between memory and time cost, and the other is optimized for fast sampling the skewed probability distributions existed in natural graphs.
Matching-oriented Product Quantization For Ad-hoc Retrieval
In this work, we propose the Matching-oriented Product Quantization (MoPQ), where a novel objective Multinoulli Contrastive Loss (MCL) is formulated.
Training Large-Scale News Recommenders with Pretrained Language Models in the Loop
Secondly, it improves the data efficiency of the training workflow, where non-informative data can be eliminated from encoding.
Efficient Automatic CASH via Rising Bandits
In this framework, the BO methods are used to solve the HPO problem for each ML algorithm separately, incorporating a much smaller hyperparameter space for BO methods.
UniNet: Scalable Network Representation Learning with Metropolis-Hastings Sampling
Finally, with the new edge sampler and random walk model abstraction, we carefully implement a scalable NRL framework called UniNet.
DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition
Despite the wide application of Graph Convolutional Network (GCN), one major limitation is that it does not benefit from the increasing depth and suffers from the oversmoothing problem.
An Experimental Evaluation of Large Scale GBDT Systems
Gradient boosting decision tree (GBDT) is a widely-used machine learning algorithm in both data analytic competitions and real-world industrial applications.
NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding
Negative sampling, which samples negative triplets from non-observed ones in the training data, is an important step in KG embedding.
Ranked #5 on
Link Prediction
on FB15k
Fast Hyperparameter Optimization of Deep Neural Networks via Ensembling Multiple Surrogates
The performance of deep neural networks crucially depends on good hyperparameter configurations.
