Search Results for author:
Found 98 papers, 55 papers with code
PatchRec: Multi-Grained Patching for Efficient LLM-based Sequential Recommendation
The framework consists of two stages: (1) Patch Pre-training, which familiarizes LLMs with item-level compression patterns, and (2) Patch Fine-tuning, which teaches LLMs to model sequences at multiple granularities.
Autonomy-of-Experts Models
We argue that the separation between the router's decision-making and the experts' execution is a critical yet overlooked issue, leading to suboptimal expert selection and ineffective learning.
Scaling Laws for Floating Point Quantization Training
Low-precision training is considered an effective strategy for reducing both training and downstream inference costs.
Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant"
Contrastive learning is a prevalent technique in self-supervised vision representation learning, typically generating positive pairs by applying two data augmentations to the same image.
DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models
Large vision-language models (LVLMs) have demonstrated exceptional performance on complex multimodal tasks.
Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
Multimodal Large Language Models (MLLMs) are known to hallucinate, which limits their practical applications.
More Expressive Attention with Negative Weights
We propose a novel attention mechanism, named Cog Attention, that enables attention weights to be negative for enhanced expressiveness, which stems from two key factors: (1) Cog Attention enhances parameter flexibility.
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens.
FairDgcl: Fairness-aware Recommendation with Dynamic Graph Contrastive Learning
Specifically, we propose FairDgcl, a dynamic graph adversarial contrastive learning framework aiming at improving fairness in recommender system.
Exploring Forgetting in Large Language Model Pre-Training
Catastrophic forgetting remains a formidable obstacle to building an omniscient model in large language models (LLMs).
Continuous Speech Tokenizer in Text To Speech
Our results show that the speech language model based on the continuous speech tokenizer has better continuity and higher estimated Mean Opinion Scores (MoS).
Lossless KV Cache Compression to 2%
Large language models have revolutionized data processing in numerous domains, with their ability to handle extended context reasoning receiving notable recognition.
RosePO: Aligning LLM-based Recommenders with Human Values
Recently, there has been a growing interest in leveraging Large Language Models (LLMs) for recommendation systems, which usually adapt a pre-trained LLM to the recommendation scenario through supervised fine-tuning (SFT).
Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions
Hallucinations in multimodal large language models (MLLMs) hinder their practical applications.
Multimodal Clickbait Detection by De-confounding Biases Using Causal Representation Inference
Considering these features are often mixed up with unknown biases, we then disentangle three kinds of latent factors from them, including the invariant factor that indicates intrinsic bait intention; the causal factor which reflects deceptive patterns in a certain scenario, and non-causal noise.
Exploring the Benefit of Activation Sparsity in Pre-training
SSD adaptively switches between the Mixtures-of-Experts (MoE) based sparse training and the conventional dense training during the pre-training process, leveraging the efficiency of sparse training and avoiding the static activation correlation of sparse training.
Language Models "Grok" to Copy
We examine the pre-training dynamics of language models, focusing on their ability to copy text from preceding context--a fundamental skill for various LLM applications, including in-context learning (ICL) and retrieval-augmented generation (RAG).
Negative Sampling in Recommendation: A Survey and Future Directions
Recommender systems aim to capture users' personalized preferences from the cast amount of user behaviors, making them pivotal in the era of information explosion.
PIP: Detecting Adversarial Examples in Large Vision-Language Models via Attention Patterns of Irrelevant Probe Questions
We propose an unconventional method named PIP, which utilizes the attention patterns of one randomly selected irrelevant probe question (e. g., "Is there a clock?")
Towards Empathetic Conversational Recommender Systems
With empathy we refer to a system's ability to capture and express emotions.
HMoE: Heterogeneous Mixture of Experts for Language Modeling
To address the imbalance in expert activation, we propose a novel training objective that encourages the frequent activation of smaller experts, enhancing computational efficiency and parameter utilization.
Modeling Domain and Feedback Transitions for Cross-Domain Sequential Recommendation
In this paper, we propose $\text{Transition}^2$, a novel method to model transitions across both domains and types of user feedback.
Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence
The rapid advancement of large language models (LLMs) has paved the way for the development of highly capable autonomous agents.
Improving Multi-modal Recommender Systems by Denoising and Aligning Multi-modal Content and User Feedback
Multi-modal recommender systems (MRSs) are pivotal in diverse online web platforms and have garnered considerable attention in recent years.
QAGCF: Graph Collaborative Filtering for Q&A Recommendation
Traditional recommendation methods treat the question-answer pair as a whole or only consider the answer as a single item, which overlooks the two challenges and cannot effectively model user interests.
DFGNN: Dual-frequency Graph Neural Network for Sign-aware Feedback
Based on the two observations, we propose a novel model that models positive and negative feedback from a frequency filter perspective called Dual-frequency Graph Neural Network for Sign-aware Recommendation (DFGNN).
ID-centric Pre-training for Recommendation
Specifically, in pre-training stage, besides the ID-based sequential model for recommendation, we also build a Cross-domain ID-matcher (CDIM) learned by both behavioral and modality information.
The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation
Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs).
Advancing LLM Reasoning Generalists with Preference Trees
We introduce Eurus, a suite of large language models (LLMs) optimized for reasoning.
Content-Based Collaborative Generation for Recommender Systems
In this paper, we propose content-based collaborative generation for recommender systems, namely ColaRec.
PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
This paper contributes a ChatGPT-Prompted visual hallucination evaluation Dataset (PhD) for objective VHE at a large scale.
Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models
Underlying data distributions of natural language, programming code, and mathematical symbols vary vastly, presenting a complex challenge for large language models (LLMs) that strive to achieve high performance across all three domains simultaneously.
Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
In practice, the multifaceted nature of human preferences inadvertently introduces what is known as the "alignment tax" -a compromise where enhancements in alignment within one objective (e. g., harmlessness) can diminish performance in others (e. g., helpfulness).
Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication
Natural language (NL) has long been the predominant format for human cognition and communication, and by extension, has been similarly pivotal in the development and application of Large Language Models (LLMs).
Plug-in Diffusion Model for Sequential Recommendation
To address this issue, this paper presents a novel Plug-in Diffusion Model for Recommendation (PDRec) framework, which employs the diffusion model as a flexible plugin to jointly take full advantage of the diffusion-generating user preferences on all items.
Curriculum-scheduled Knowledge Distillation from Multiple Pre-trained Teachers for Multi-domain Sequential Recommendation
Pre-trained recommendation models (PRMs) have received increasing interest recently.
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships.
Universal Multi-modal Multi-domain Pre-trained Recommendation
There is a rapidly-growing research interest in modeling user preferences via pre-training multi-domain interactions for recommender systems.
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
Pre-trained language models (PLMs) have achieved remarkable results on NLP tasks but at the expense of huge parameter sizes and the consequent computational costs.
Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language
With the thriving of pre-trained language model (PLM) widely verified in various of NLP tasks, pioneer efforts attempt to explore the possible cooperation of the general textual information in PLM with the personalized behavioral information in user historical behavior sequences to enhance sequential recommendation (SR).
Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models
Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise.
AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems
The optimized agents can also propagate their preferences to other agents in subsequent interactions, implicitly capturing the collaborative filtering idea.
UltraFeedback: Boosting Language Models with Scaled AI Feedback
Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models, serving as a solid foundation for future feedback learning research.
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
Autonomous agents empowered by Large Language Models (LLMs) have undergone significant improvements, enabling them to generalize across a broad spectrum of tasks.
Learning from All Sides: Diversified Positive Augmentation via Self-distillation in Recommendation
A powerful positive item augmentation is beneficial to address the sparsity issue, while few works could jointly consider both the accuracy and diversity of these augmented training labels.
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Based on ToolBench, we fine-tune LLaMA to obtain an LLM ToolLLaMA, and equip it with a neural API retriever to recommend appropriate APIs for each instruction.
Ranked #3 on
Trajectory Planning
on ToolBench
Emergent Modularity in Pre-trained Transformers
In analogy to human brains, we consider two main characteristics of modularity: (1) functional specialization of neurons: we evaluate whether each neuron is mainly specialized in a certain function, and find that the answer is yes.
Large Language Models are Zero-Shot Rankers for Recommender Systems
Recently, large language models (LLMs) (e. g., GPT-4) have demonstrated impressive general-purpose task-solving abilities, including the potential to approach recommendation tasks.
Recyclable Tuning for Continual Pre-training
In pilot studies, we find that after continual pre-training, the upgraded PLM remains compatible with the outdated adapted weights to some extent.
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach
Inspired by the recent progress on large language models (LLMs), we take a different approach to developing the recommendation models, considering recommendation as instruction following by LLMs.
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
We recruit annotators to search for relevant information using our interface and then answer questions.
Attacking Pre-trained Recommendation
Recently, a series of pioneer studies have shown the potency of pre-trained models in sequential recommendation, illuminating the path of building an omniscient unified pre-trained recommendation model for different downstream recommendation tasks.
Triple Sequence Learning for Cross-domain Recommendation
To address this issue, we present a novel framework, termed triple sequence learning for cross-domain recommendation (Tri-CDR), which jointly models the source, target, and mixed behavior sequences to highlight the global and target preference and precisely model the triple correlation in CDR.
A Survey on Causal Inference for Recommendation
Considering RS researchers' unfamiliarity with causality, it is necessary yet challenging to comprehensively review relevant studies from a coherent causal theoretical perspective, thereby facilitating a deeper integration of causal inference in RS.
Adversarial Learning Data Augmentation for Graph Contrastive Learning in Recommendation
To reduce the influence of data sparsity, Graph Contrastive Learning (GCL) is adopted in GNN-based CF methods for enhancing performance.
Visually Grounded Commonsense Knowledge Acquisition
In this work, we present CLEVER, which formulates CKE as a distantly supervised multi-instance learning problem, where models learn to summarize commonsense relations from a bag of images about an entity pair without any human annotation on image instances.
Pruning Pre-trained Language Models Without Fine-Tuning
These methods, such as movement pruning, use first-order information to prune PLMs while fine-tuning the remaining weights.
Better Pre-Training by Reducing Representation Confusion
Motivated by the above investigation, we propose two novel techniques to improve pre-trained language models: Decoupled Directional Relative Position (DDRP) encoding and MTH pre-training objective.
Reweighting Clicks with Dwell Time in Recommendation
Precisely, we first define a new behavior named valid read, which helps to select high-quality click instances for different users and items via dwell time.
Multi-granularity Item-based Contrastive Recommendation
Specifically, we build three item-based CL tasks as a set of plug-and-play auxiliary objectives to capture item correlations in feature, semantic and session levels.
Customized Conversational Recommender Systems
In this paper, we propose two key points for CRS to improve the user experience: (1) Speaking like a human, human can speak with different styles according to the current dialogue context.
Prompt Tuning for Discriminative Pre-trained Language Models
Recent works have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing (NLP) tasks.
Personalized Prompt for Sequential Recommendation
Specifically, we build the personalized soft prefix prompt via a prompt generator based on user profiles and enable a sufficient training of prompts via a prompt-oriented contrastive learning with both prompt- and behavior-based augmentations.
User-Centric Conversational Recommendation with Multi-Aspect User Modeling
In this work, we highlight that the user's historical dialogue sessions and look-alike users are essential sources of user preferences besides the current dialogue session in CRS.
Ranked #3 on
Recommendation Systems
on ReDial
(Recall@50 metric)
Multi-view Multi-behavior Contrastive Learning in Recommendation
We argue that MBR models should: (1) model the coarse-grained commonalities between different behaviors of a user, (2) consider both individual sequence view and global graph view in multi-behavior modeling, and (3) capture the fine-grained differences between multiple behaviors of a user.
Contrastive Cross-domain Recommendation in Matching
Cross-domain recommendation (CDR) aims to provide better recommendation results in the target domain with the help of the source domain, which is widely used and explored in real-world systems.
Curriculum Disentangled Recommendation with Noisy Multi-feedback
However, learning such disentangled representations from multi-feedback data is challenging because i) multi-feedback is complex: there exist complex relations among different types of feedback (e. g., click, unclick, and dislike, etc) as well as various user intentions, and ii) multi-feedback is noisy: there exists noisy (useless) information both in features and labels, which may deteriorate the recommendation performance.
MIC: Model-agnostic Integrated Cross-channel Recommenders
However, existing work are primarily built upon pre-defined retrieval channels, including User-CF (U2U), Item-CF (I2I), and Embedding-based Retrieval (U2I), thus access to the limited correlation between users and items which solely entail from partial information of latent interactions.
Personalized Transfer of User Preferences for Cross-domain Recommendation
Specifically, a meta network fed with users' characteristic embeddings is learned to generate personalized bridge functions to achieve personalized transfer of preferences for each user.
USER: A Unified Information Search and Recommendation Model based on Integrated Behavior Sequence
Search and recommendation are the two most common approaches used by people to obtain information.
Open Hierarchical Relation Extraction
To establish the bidirectional connections between OpenRE and relation hierarchy, we propose the task of open hierarchical relation extraction and present a novel OHRE framework for the task.
Learning to Expand Audience via Meta Hybrid Experts and Critics for Recommendation and Advertising
Besides, MetaHeac has been successfully deployed in WeChat for the promotion of both contents and advertisements, leading to great improvement in the quality of marketing.
Transfer-Meta Framework for Cross-domain Recommendation to Cold-Start Users
With the advantage of meta learning which has good generalization ability to novel tasks, we propose a transfer-meta framework for CDR (TMCDR) which has a transfer stage and a meta stage.
Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks
The cold item ID embedding has two main problems: (1) A gap is existing between the cold ID embedding and the deep model.
Long Short-Term Temporal Meta-learning in Online Recommendation
An effective online recommendation system should jointly capture users' long-term and short-term preferences in both users' internal behaviors (from the target recommendation task) and external behaviors (from other tasks).
Real-time Relevant Recommendation Suggestion
When finishing reading an item, users may want to access more relevant items related to the last read one as extended reading.
Understanding WeChat User Preferences and "Wow" Diffusion
"Top Stories" is a novel friend-enhanced recommendation engine in WeChat, in which users can read articles based on preferences of both their own and their friends.
Graph Representation Learning
Social and Information Networks
UPRec: User-Aware Pre-training for Recommender Systems
Existing sequential recommendation methods rely on large amounts of training data and usually suffer from the data sparsity problem.
Improving Accuracy and Diversity in Matching of Recommendation with Diversified Preference Network
In this paper, we propose a novel Heterogeneous graph neural network framework for diversified recommendation (GraphDR) in matching to improve both recommendation accuracy and diversity.
Meta-Information Guided Meta-Learning for Few-Shot Relation Classification
Few-shot classification requires classifiers to adapt to new classes with only a few training instances.
Denoising Relation Extraction from Document-level Distant Supervision
Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance.
Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect
Recommender systems aim to provide item recommendations for users, and are usually faced with data sparsity problem (e. g., cold start) in real-world scenarios.
Connecting Embeddings for Knowledge Graph Entity Typing
In this paper, we propose a novel approach for KG entity typing which is trained by jointly utilizing local typing knowledge from existing entity type assertions and global triple knowledge from KGs.
FAQ-based Question Answering via Knowledge Anchors
In this paper, we propose a novel Knowledge Anchor based Question Answering (KAQA) framework for FAQ-based QA to better understand questions and retrieve more appropriate answers.
Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data
Open relation extraction (OpenRE) aims to extract relational facts from the open-domain corpus.
Neural Snowball for Few-Shot Relation Learning
To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations.
Knowledge Representation Learning: A Quantitative Review
Knowledge representation learning (KRL) aims to represent entities and relations in knowledge graph in low-dimensional semantic space, which have been widely used in massive knowledge-driven tasks.
Language Modeling with Sparse Product of Sememe Experts
Most language modeling methods rely on large-scale data to statistically learn the sequential patterns of words.
Cross-lingual Lexical Sememe Prediction
We propose a novel framework to model correlations between sememes and multi-lingual words in low-dimensional semantic space for sememe prediction.
Incorporating Chinese Characters of Words for Lexical Sememe Prediction
However, existing methods of lexical sememe prediction typically rely on the external context of words to represent the meaning, which usually fails to deal with low-frequency and out-of-vocabulary words.
Improved Word Representation Learning with Sememes
The key idea is to utilize word sememes to capture exact meanings of a word within specific contexts accurately.
Iterative Entity Alignment via Joint Knowledge Embeddings
During this process, we can align entities according to their semantic distance in this joint semantic space.
Does William Shakespeare REALLY Write Hamlet? Knowledge Representation Learning with Confidence
Experimental results demonstrate that our confidence-aware models achieve significant and consistent improvements on all tasks, which confirms the capability of CKRL modeling confidence with structural information in both KG noise detection and knowledge representation learning.
Neural Emoji Recommendation in Dialogue Systems
It indicates that our method could well capture the contextual information and emotion flow in dialogues, which is significant for emoji recommendation.
Image-embodied Knowledge Representation Learning
More specifically, we first construct representations for all images of an entity with a neural image encoder.
Knowledge Representation via Joint Learning of Sequential Text and Knowledge Graphs
There are two main challenges for constructing knowledge representations from plain texts: (1) How to take full advantages of sequential contexts of entities in plain texts for KRL.
