Search Results for author:
Found 68 papers, 44 papers with code
Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning
Inspired by the remarkable reasoning capabilities of Deepseek-R1 in complex textual tasks, many works attempt to incentivize similar capabilities in Multimodal Large Language Models (MLLMs) by directly applying reinforcement learning (RL).
Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning
While showing sophisticated reasoning abilities, large language models (LLMs) still struggle with long-horizon decision-making tasks due to deficient exploration and long-term credit assignment, especially in sparse-reward scenarios.
Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models
Large Multimodal Models(LMMs) face notable challenges when encountering multimodal knowledge conflicts, particularly under retrieval-augmented generation(RAG) frameworks where the contextual information from external sources may contradict the model's internal parametric knowledge, leading to unreliable outputs.
SATORI-R1: Incentivizing Multimodal Reasoning with Spatial Grounding and Verifiable Rewards
Our analysis of the attention map confirms enhanced focus on critical regions, which brings improvements in accuracy.
Step-level Reward for Free in RL-based T2I Diffusion Model Fine-tuning
Recent advances in text-to-image (T2I) diffusion model fine-tuning leverage reinforcement learning (RL) to align generated images with learnable reward functions.
Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models
Instruction-following is essential for aligning large language models (LLMs) with user intent.
OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning
We hope OpenThinkIMG can serve as a foundational framework for advancing dynamic, tool-augmented visual reasoning, helping the community develop AI agents that can genuinely "think with images".
Learning to Reason under Off-Policy Guidance
Recent advances in large reasoning models (LRMs) demonstrate that sophisticated behaviors such as multi-step reasoning and self-reflection can emerge via reinforcement learning with verifiable rewards~(\textit{RLVR}).
SEE: Continual Fine-tuning with Sequential Ensemble of Experts
Some approaches use routers to assign tasks to experts, but in continual learning, they often require retraining for optimal performance.
A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
Recent Large Reasoning Models (LRMs), such as DeepSeek-R1 and OpenAI o1, have demonstrated strong performance gains by scaling up the length of Chain-of-Thought (CoT) reasoning during inference.
From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibration
Despite this success, the training data of LVLMs still suffers from Long-Tail (LT) problems, where the data distribution is highly imbalanced.
Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
Linear-MoE leverages the advantages of both LSM modules for linear-complexity sequence modeling and MoE layers for sparsely activation, aiming to offer high performance with efficient training.
Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think
(3) With the above two-stage models excelling in motion controllability and degree, we decouple the relevant parameters associated with each type of motion ability and inject them into the base I2V-DM.
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
While Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning for Large Language Models (LLMs), its performance often falls short of Full Fine-Tuning (Full FT).
LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid
In this paper, we introduce LASP-2, a new SP method to enhance both communication and computation parallelism when training linear attention transformer models with very-long input sequences.
Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
In this work, we introduce Test-time Preference Optimization (TPO), a framework that aligns LLM outputs with human preferences during inference, removing the need to update model parameters.
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Since language models are prone to various types of errors during the reasoning process, PRMs are required to possess nuanced capabilities for detecting various implicit error types in real-world scenarios.
Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraints
Diffusion Transformers (DiT) have emerged as a powerful architecture for image and video generation, offering superior quality and scalability.
LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
Recently, inspired by the concept of sparsity, Mixture-of-Experts (MoE) models have gained increasing popularity for scaling model size while keeping the number of activated parameters constant.
CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
In recent years, Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in multimodal intelligence.
SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information
To address these challenges, we propose a self-refinement framework designed to teach LVLMs to Selectively Utilize Retrieved Information (SURf).
Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
By fully grasping the information in the image and carefully considering the certainty of the potential answers when decoding, our MVP can effectively reduce hallucinations in LVLMs. The extensive experiments verify that our proposed MVP significantly mitigates the hallucination problem across four well-known LVLMs.
ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
Only a few research explored the conflicts between the inherent knowledge of LLMs and the retrieved contextual knowledge.
Mitigating Multilingual Hallucination in Large Vision-Language Models
In this paper, we make the first attempt to mitigate this important multilingual hallucination in LVLMs.
Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation
To assess the capability of our proposed ARA model in reducing hallucination, we employ three widely used LVLM models (LLaVA-1. 5, Qwen-VL, and mPLUG-Owl2) across four benchmarks.
A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
Compared to traditional Large Language Models (LLMs), LVLMs present great potential and challenges due to its closer proximity to the multi-resource real-world applications and the complexity of multi-modal processing.
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
Motivated by this limit, we investigate building MoE models from existing dense large language models.
Timo: Towards Better Temporal Reasoning for Language Models
Therefore, we propose a crucial question: Can we build a universal framework to handle a variety of temporal reasoning tasks?
Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts
Mixture-of-Experts (MoE) models have shown remarkable capability in instruction tuning, especially when the number of tasks scales.
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion
\thm{Can we fine-tune a series of task-specific small models and transfer their knowledge directly to a much larger model without additional training?}
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
In view of this, we propose Twin-Merging, a method that encompasses two principal stages: (1) modularizing knowledge into shared and exclusive components, with compression to reduce redundancy and enhance efficiency; (2) dynamically merging shared and task-specific knowledge based on the input.
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?
Temporal reasoning is fundamental for large language models (LLMs) to comprehend the world.
Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models
Our approach is grounded in the empirical observation that pairwise comparisons can effectively alleviate boundary ambiguity and inherent bias.
Improving Pseudo Labels with Global-Local Denoising Framework for Cross-lingual Named Entity Recognition
In this paper, we propose a Global-Local Denoising framework (GLoDe) for cross-lingual NER.
A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching
Although DC-Match is a simple yet effective method for semantic matching, it highly depends on the external NER techniques to identify the keywords of sentences, which limits the performance of semantic matching for minor languages since satisfactory NER tools are usually hard to obtain.
Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning
Self-Supervised Contrastive Learning has proven effective in deriving high-quality representations from unlabeled data.
Joint Multi-Facts Reasoning Network For Complex Temporal Question Answering Over Knowledge Graph
Temporal Knowledge Graph (TKG) is an extension of regular knowledge graph by attaching the time scope.
Enhancing Low-Resource Relation Representations through Multi-View Decoupling
Recently, prompt-tuning with pre-trained language models (PLMs) has demonstrated the significantly enhancing ability of relation extraction (RE) tasks.
Mirror: A Universal Framework for Various Information Extraction Tasks
Sharing knowledge between information extraction tasks has always been a challenge due to the diverse data formats and task variations.
Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding
In this manner, our framework is able to learn the unified representations of uni-modal or multi-modal skeleton input, which is flexible to different kinds of modality input for robust action understanding in practical cases.
MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control
Subsequently, we employ a conditional variational auto-encoder to align with the dense personalized responses within a latent joint attribute space.
TREA: Tree-Structure Reasoning Schema for Conversational Recommendation
TREA constructs a multi-hierarchical scalable tree as the reasoning structure to clarify the causal relationships between mentioned entities, and fully utilizes historical conversations to generate more reasonable and suitable responses for recommended results.
From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval
As the attribute-specific similarity typically corresponds to the specific subtle regions of images, we propose a Region-to-Patch Framework (RPF) that consists of a region-aware branch and a patch-aware branch to extract fine-grained attribute-related visual features for precise retrieval in a coarse-to-fine manner.
Transform-Equivariant Consistency Learning for Temporal Sentence Grounding
This paper addresses the temporal sentence grounding (TSG).
A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trends
In this paper, we provide a comprehensive review of the development of Arabic NER, especially the recent advances in deep learning and pre-trained language model.
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval
During the knowledge distillation, an inheritance student branch is devised to absorb the knowledge from the teacher model.
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble
Distantly-Supervised Named Entity Recognition (DS-NER) effectively alleviates the data scarcity problem in NER by automatically generating training samples.
Reducing the Vision and Language Bias for Temporal Sentence Grounding
In this paper, we study the above issue of selection biases and accordingly propose a Debiasing-TSG (D-TSG) model to filter and remove the negative biases in both vision and language modalities for enhancing the model generalization ability.
Delving Deep into Regularity: A Simple but Effective Method for Chinese Named Entity Recognition
Recent years have witnessed the improving performance of Chinese Named Entity Recognition (NER) from proposing new frameworks or incorporating word lexicons.
Chinese Named Entity Recognition
named-entity-recognition
+3
Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval
In this work, we concentrate on video representation learning, an essential component for text-to-video retrieval.
Unsupervised Temporal Video Grounding with Deep Semantic Clustering
Temporal video grounding (TVG) aims to localize a target segment in a video according to a given sentence query.
Exploring Motion and Appearance Information for Temporal Sentence Grounding
Then, we develop separate motion and appearance branches to learn motion-guided and appearance-guided object relations, respectively.
Memory-Guided Semantic Learning Network for Temporal Sentence Grounding
To tackle this issue, we propose a memory-augmented network, called Memory-Guided Semantic Learning Network (MGSL-Net), that learns and memorizes the rarely appeared content in TSG tasks.
Efficient Document-level Event Extraction via Pseudo-Trigger-aware Pruned Complete Graph
Most previous studies of document-level event extraction mainly focus on building argument chains in an autoregressive way, which achieves a certain success but is inefficient in both training and inference.
Ranked #3 on
Document-level Event Extraction
on ChFinAnn
Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding
A key solution to temporal sentence grounding (TSG) exists in how to learn effective alignment between vision and language features extracted from an untrimmed video and a sentence description.
Adaptive Proposal Generation Network for Temporal Sentence Localization in Videos
However, the performance of bottom-up model is inferior to the top-down counterpart as it fails to exploit the segment-level interaction.
Coarse to Fine: Domain Adaptive Crowd Counting via Adversarial Scoring Network
In specific, at the coarse-grained stage, we design a dual-discriminator strategy to adapt source domain to be close to the targets from the perspectives of both global and local feature space via adversarial learning.
Context-aware Biaffine Localizing Network for Temporal Sentence Grounding
This paper addresses the problem of temporal sentence grounding (TSG), which aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query.
Hierarchical Similarity Learning for Language-based Product Image Retrieval
In this paper, we focus on the cross-modal similarity measurement, and propose a novel Hierarchical Similarity Learning (HSL) network.
Progressive Localization Networks for Language-based Moment Localization
The language-based setting of this task allows for an open set of target activities, resulting in a large variation of the temporal lengths of video moments.
Read, Retrospect, Select: An MRC Framework to Short Text Entity Linking
Entity linking (EL) for the rapidly growing short text (e. g. search queries and news titles) is critical to industrial applications.
Reasoning Step-by-Step: Temporal Sentence Localization in Videos via Deep Rectification-Modulation Network
In this paper, we propose a novel deep rectification-modulation network (RMN), transforming this task into a multi-step reasoning process by repeating rectification and modulation.
Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videos
In this paper, we propose a Fine-grained Iterative Attention Network (FIAN) that consists of an iterative attention module for bilateral query-video in-formation extraction.
Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization
To this end, we propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph.
Enhanced 3D convolutional networks for crowd counting
Recently, convolutional neural networks (CNNs) are the leading defacto method for crowd counting.
Attend To Count: Crowd Counting with Adaptive Capacity Multi-scale CNNs
ACM-CNN consists of three types of modules: a coarse network, a fine network, and a smooth network.
Adversarial Category Alignment Network for Cross-domain Sentiment Classification
Cross-domain sentiment classification aims to predict sentiment polarity on a target domain utilizing a classifier learned from a source domain.
