Search Results for author:
Found 37 papers, 12 papers with code
PatchRec: Multi-Grained Patching for Efficient LLM-based Sequential Recommendation
The framework consists of two stages: (1) Patch Pre-training, which familiarizes LLMs with item-level compression patterns, and (2) Patch Fine-tuning, which teaches LLMs to model sequences at multiple granularities.
Autonomy-of-Experts Models
We argue that the separation between the router's decision-making and the experts' execution is a critical yet overlooked issue, leading to suboptimal expert selection and ineffective learning.
Scaling Laws for Floating Point Quantization Training
Low-precision training is considered an effective strategy for reducing both training and downstream inference costs.
Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant"
Contrastive learning is a prevalent technique in self-supervised vision representation learning, typically generating positive pairs by applying two data augmentations to the same image.
DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models
Large vision-language models (LVLMs) have demonstrated exceptional performance on complex multimodal tasks.
Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
Multimodal Large Language Models (MLLMs) are known to hallucinate, which limits their practical applications.
More Expressive Attention with Negative Weights
We propose a novel attention mechanism, named Cog Attention, that enables attention weights to be negative for enhanced expressiveness, which stems from two key factors: (1) Cog Attention enhances parameter flexibility.
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens.
Continuous Speech Tokenizer in Text To Speech
Our results show that the speech language model based on the continuous speech tokenizer has better continuity and higher estimated Mean Opinion Scores (MoS).
Exploring Forgetting in Large Language Model Pre-Training
Catastrophic forgetting remains a formidable obstacle to building an omniscient model in large language models (LLMs).
Lossless KV Cache Compression to 2%
Large language models have revolutionized data processing in numerous domains, with their ability to handle extended context reasoning receiving notable recognition.
RosePO: Aligning LLM-based Recommenders with Human Values
Recently, there has been a growing interest in leveraging Large Language Models (LLMs) for recommendation systems, which usually adapt a pre-trained LLM to the recommendation scenario through supervised fine-tuning (SFT).
Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions
Hallucinations in multimodal large language models (MLLMs) hinder their practical applications.
Language Models "Grok" to Copy
We examine the pre-training dynamics of language models, focusing on their ability to copy text from preceding context--a fundamental skill for various LLM applications, including in-context learning (ICL) and retrieval-augmented generation (RAG).
Negative Sampling in Recommendation: A Survey and Future Directions
Recommender systems aim to capture users' personalized preferences from the cast amount of user behaviors, making them pivotal in the era of information explosion.
Towards Empathetic Conversational Recommender Systems
With empathy we refer to a system's ability to capture and express emotions.
HMoE: Heterogeneous Mixture of Experts for Language Modeling
To address the imbalance in expert activation, we propose a novel training objective that encourages the frequent activation of smaller experts, enhancing computational efficiency and parameter utilization.
Diverse and Fine-Grained Instruction-Following Ability Exploration with Synthetic Data
Instruction-following is particularly crucial for large language models (LLMs) to support diverse user requests.
Improving Multi-modal Recommender Systems by Denoising and Aligning Multi-modal Content and User Feedback
Multi-modal recommender systems (MRSs) are pivotal in diverse online web platforms and have garnered considerable attention in recent years.
DFGNN: Dual-frequency Graph Neural Network for Sign-aware Feedback
Based on the two observations, we propose a novel model that models positive and negative feedback from a frequency filter perspective called Dual-frequency Graph Neural Network for Sign-aware Recommendation (DFGNN).
ID-centric Pre-training for Recommendation
Specifically, in pre-training stage, besides the ID-based sequential model for recommendation, we also build a Cross-domain ID-matcher (CDIM) learned by both behavioral and modality information.
The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation
Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs).
PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
This paper contributes a ChatGPT-Prompted visual hallucination evaluation Dataset (PhD) for objective VHE at a large scale.
EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
To our best knowledge, we are the first work that achieves almost lossless quantization performance for LLMs under a data-independent setting and our algorithm runs over 10 times faster than the data-dependent methods.
Plug-in Diffusion Model for Sequential Recommendation
To address this issue, this paper presents a novel Plug-in Diffusion Model for Recommendation (PDRec) framework, which employs the diffusion model as a flexible plugin to jointly take full advantage of the diffusion-generating user preferences on all items.
Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes.
E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
Traditional pruning methods are known to be challenging to work in Large Language Models (LLMs) for Generative AI because of their unaffordable training process and large computational demands.
Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language
With the thriving of pre-trained language model (PLM) widely verified in various of NLP tasks, pioneer efforts attempt to explore the possible cooperation of the general textual information in PLM with the personalized behavioral information in user historical behavior sequences to enhance sequential recommendation (SR).
TeachCLIP: Multi-Grained Teaching for Efficient Text-to-Video Retrieval
For efficient T2VR, we propose TeachCLIP with multi-grained teaching to let a CLIP4Clip based student network learn from more advanced yet computationally heavy models such as X-CLIP, TS2-Net and X-Pool .
Pretraining De-Biased Language Model with Large-scale Click Logs for Document Ranking
Pre-trained language models have achieved great success in various large-scale information retrieval tasks.
Multi-Feature Integration for Perception-Dependent Examination-Bias Estimation
Eliminating examination bias accurately is pivotal to apply click-through data to train an unbiased ranking model.
BagFormer: Better Cross-Modal Retrieval via bag-wise interaction
In the field of cross-modal retrieval, single encoder models tend to perform better than dual encoder models, but they suffer from high latency and low throughput.
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.
MKQ-BERT: Quantized BERT with 4-bits Weights and Activations
In this work, we propose MKQ-BERT, which further improves the compression level and uses 4-bits for quantization.
TexSmart: A System for Enhanced Natural Language Understanding
This paper introduces TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis
This technique report introduces TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
STH: Spatio-Temporal Hybrid Convolution for Efficient Action Recognition
Such a design enables efficient spatio-temporal modeling and maintains a small model scale.
