Search Results for author:
Found 27 papers, 21 papers with code
Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
By examining token entropy patterns in Chain-of-Thought (CoT) reasoning, we observe that only a small fraction of tokens exhibit high entropy, and these tokens act as critical forks that steer the model toward diverse reasoning pathways.
WorldPM: Scaling Human Preference Modeling
Motivated by scaling laws in language modeling that demonstrate how test loss scales as a power law with model and dataset sizes, we find that similar laws exist in preference modeling.
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Gating mechanisms have been widely utilized, from early models like LSTMs and Highway Networks to recent state space models, linear attention, and also softmax attention.
Qwen2.5-1M Technical Report
By leveraging our inference framework, the Qwen2. 5-1M models achieve a remarkable 3x to 7x prefill speedup in scenarios with 1 million tokens of context.
Channel Sounding Using Multiplicative Arrays Based on Successive Interference Cancellation Principle
Both numerical simulations and experimental validation results are provided to demonstrate the effectiveness and robustness of the proposed SIC algorithm for the MA.
Revolutionizing Encrypted Traffic Classification with MH-Net: A Multi-View Heterogeneous Graph Model
With the growing significance of network security, the classification of encrypted traffic has emerged as an urgent challenge.
Qwen2.5 Technical Report
In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2. 5-Turbo and Qwen2. 5-Plus, both available from Alibaba Cloud Model Studio.
Ranked #7 on
on GPQA
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models
Post-training has emerged as a crucial paradigm for adapting large-scale pre-trained models to various tasks, whose effects are fully reflected by delta parameters (i. e., the disparity between post-trained and pre-trained parameters).
One Train for Two Tasks: An Encrypted Traffic Classification Framework Using Supervised Contrastive Learning
In particular, we utilize supervised contrastive learning to enhance the packet-level and flow-level representations and perform graph data augmentation on the byte-level traffic graph so that the fine-grained semantic-invariant characteristics between bytes can be captured through contrastive learning.
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
We experiment with encoder- and decoder-based LMs, showing that: (1) SFT delta parameter value ranges are typically small (within 0. 002) with extreme redundancy, and DARE can effortlessly eliminate 90% or even 99% of them; (2) DARE can merge multiple task-specific LMs into one LM with diverse capabilities.
A review of individual tree crown detection and delineation from optical remote sensing images
); (2) traditional machine learning methods (such as random forest, decision tree, etc.
Pretraining Language Models with Text-Attributed Heterogeneous Graphs
In many real-world scenarios (e. g., academic networks, social platforms), different types of entities are not only associated with texts but also connected by various relationships, which can be abstracted as Text-Attributed Heterogeneous Graphs (TAHGs).
Cluster Language Model for Improved E-Commerce Retrieval and Ranking: Leveraging Query Similarity and Fine-Tuning for Personalized Results
This paper proposes a novel method to improve the accuracy of product search in e-commerce by utilizing a cluster language model.
A Simple Framework for Multi-mode Spatial-Temporal Data Modeling
Spatial-temporal data modeling aims to mine the underlying spatial relationships and temporal dependencies of objects in a system.
Adaptive Taxonomy Learning and Historical Patterns Modelling for Patent Classification
Finally, we combine the contextual information of patent texts that contains the semantics of IPC codes, and assignees' sequential preferences to make predictions.
Event-based Dynamic Graph Representation Learning for Patent Application Trend Prediction
Finally, the patent application trend is predicted by aggregating the representations of the target company and classification codes from static, dynamic, and hierarchical perspectives.
TFE-GNN: A Temporal Fusion Encoder Using Graph Neural Networks for Fine-grained Encrypted Traffic Classification
Encrypted traffic classification is receiving widespread attention from researchers and industrial companies.
An Empirical Evaluation of Temporal Graph Benchmark
In this paper, we conduct an empirical evaluation of Temporal Graph Benchmark (TGB) by extending our Dynamic Graph Library (DyGLib) to TGB.
Towards Better Dynamic Graph Learning: New Architecture and Unified Library
We propose DyGFormer, a new Transformer-based architecture for dynamic graph learning.
A Fine-grained Chinese Software Privacy Policy Dataset for Sequence Labeling and Regulation Compliant Identification
Privacy protection raises great attention on both legal levels and user awareness.
Label-Enhanced Graph Neural Network for Semi-supervised Node Classification
In recent years, several methods have been designed to additionally utilize the labels at the input.
Ranked #20 on
Node Property Prediction
on ogbn-mag
Continuous-Time User Preference Modelling for Temporal Sets Prediction
Previous studies for temporal sets prediction mainly focus on the modelling of elements and implicitly represent each user's preference based on his/her interacted elements.
Heterogeneous Graph Representation Learning with Relation Awareness
Moreover, a semantic fusing module is presented to aggregate relation-aware node representations into a compact representation with the learned relation representations.
Ranked #23 on
Node Property Prediction
on ogbn-mag
Hybrid Micro/Macro Level Convolution for Heterogeneous Graph Learning
Representation learning on heterogeneous graphs aims to obtain low-dimensional node representations that could preserve both node attributes and relation information.
Ranked #25 on
Node Property Prediction
on ogbn-mag
Cross-regional oil palm tree counting and detection via multi-level attention domain adaptation network
In this paper, we propose a novel domain adaptive oil palm tree detection method, i. e., a Multi-level Attention Domain Adaptation Network (MADAN) to reap cross-regional oil palm tree counting and detection.
Predicting Temporal Sets with Deep Neural Networks
Given a sequence of sets, where each set contains an arbitrary number of elements, the problem of temporal sets prediction aims to predict the elements in the subsequent set.
