Search Results for author:
Found 40 papers, 13 papers with code
PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC
From the decision-making perspective, to handle complex user instructions and interdependent subtasks more effectively, we propose a hierarchical multi-agent collaboration architecture that decomposes decision-making processes into Instruction-Subtask-Action levels.
SA-GNAS: Seed Architecture Expansion for Efficient Large-scale Graph Neural Architecture Search
GNAS (Graph Neural Architecture Search) has demonstrated great effectiveness in automatically designing the optimal graph neural architectures for multiple downstream tasks, such as node classification and link prediction.
mR$^2$AG: Multimodal Retrieval-Reflection-Augmented Generation for Knowledge-Based VQA
Thus, multimodal Retrieval-Augmented Generation (mRAG) is naturally introduced to provide MLLMs with comprehensive and up-to-date knowledge, effectively expanding the knowledge scope.
SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning
The quality analysis of ROIs can provide fine-grained guidance for image quality improvement and is crucial for scenarios focusing on region-level quality.
Learning from Neighbors: Category Extrapolation for Long-Tail Learning
To prevent the overwhelming presence of auxiliary classes from disrupting training, we introduce a neighbor-silencing loss that encourages the model to focus on class discrimination within the target dataset.
MIBench: Evaluating Multimodal Large Language Models over Multiple Images
However, most existing MLLMs and benchmarks primarily focus on single-image input scenarios, leaving the performance of MLLMs when handling realistic multiple images underexplored.
EA-VTR: Event-Aware Video-Text Retrieval
EA-VTR can efficiently encode frame-level and video-level visual representations simultaneously, enabling detailed event content and complex event temporal cross-modal alignment, ultimately enhancing the comprehensive understanding of video events.
How to Make Cross Encoder a Good Teacher for Efficient Image-Text Retrieval?
Dominant dual-encoder models enable efficient image-text retrieval but suffer from limited accuracy while the cross-encoder models offer higher accuracy at the expense of efficiency.
PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts
On the other hand, PromptIQA is trained on a mixed dataset with two proposed data augmentation strategies to learn diverse requirements, thus enabling it to effectively adapt to new requirements.
Semantics-enhanced Cross-modal Masked Image Modeling for Vision-Language Pre-training
In vision-language pre-training (VLP), masked image modeling (MIM) has recently been introduced for fine-grained cross-modal alignment.
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval
In this work, we propose the UNIFY framework, which learns lexicon representations to capture fine-grained semantics and combines the strengths of latent and lexicon representations for video-text retrieval.
GMC-IQA: Exploiting Global-correlation and Mean-opinion Consistency for No-reference Image Quality Assessment
Due to the subjective nature of image quality assessment (IQA), assessing which image has better quality among a sequence of images is more reliable than assigning an absolute mean opinion score for an image.
Set Prediction Guided by Semantic Concepts for Diverse Video Captioning
Each caption in the set is attached to a concept combination indicating the primary semantic content of the caption and facilitating element alignment in set prediction.
Simple and Efficient Partial Graph Adversarial Attack: A New Perspective
To this end, we propose a totally new method named partial graph attack (PGA), which selects the vulnerable nodes as attack targets.
HAGNN: Hybrid Aggregation for Heterogeneous Graph Neural Networks
Then, we propose a novel framework to utilize the rich type semantic information in heterogeneous graphs comprehensively, namely HAGNN (Hybrid Aggregation for Heterogeneous GNNs).
AutoAC: Towards Automated Attribute Completion for Heterogeneous Graph Neural Network
Moreover, to improve the performance of the downstream graph learning task, attribute completion and the training of the heterogeneous GNN should be jointly optimized rather than viewed as two separate processes.
Order-Prompted Tag Sequence Generation for Video Tagging
This paper proposes a novel generative model, Order-Prompted Tag Sequence Generation (OP-TSG), according to the above characteristics.
ViLEM: Visual-Language Error Modeling for Image-Text Retrieval
ViLEM then enforces the model to discriminate the correctness of each word in the plausible negative texts and further correct the wrong words via resorting to image information.
Ranked #45 on
Visual Reasoning
on Winoground
Knowledge-enhanced Black-box Attacks for Recommendations
Recent studies have shown that deep neural networks-based recommender systems are vulnerable to adversarial attacks, where attackers can inject carefully crafted fake user profiles (i. e., a set of items that fake users have interacted with) into a target recommender system to achieve malicious purposes, such as promote or demote a set of target items.
PIC 4th Challenge: Semantic-Assisted Multi-Feature Encoding and Multi-Head Decoding for Dense Video Captioning
The task of Dense Video Captioning (DVC) aims to generate captions with timestamps for multiple events in one video.
Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoning
They base the visual grounding on the features from pre-generated proposals or anchors, and fuse these features with the text embeddings to locate the target mentioned by the text.
CREATE: A Benchmark for Chinese Short Video Retrieval and Title Generation
In this paper, we propose to CREATE, the first large-scale Chinese shoRt vidEo retrievAl and Title gEneration benchmark, to facilitate research and application in video titling and video retrieval in Chinese.
Transition Relation Aware Self-Attention for Session-based Recommendation
Specifically, TRASA first converts the session to a graph and then encodes the shortest path between items through the gated recurrent unit as their transition relation.
EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matching
The datasets will be released to facilitate the development of video captioning metrics.
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
Graph convolutional networks (GCNs) have been widely used and achieved remarkable results in skeleton-based action recognition.
Ranked #11 on
Skeleton Based Action Recognition
on N-UCLA
PDNet: Toward Better One-Stage Object Detection With Prediction Decoupling
However, the most suitable positions for inferring different targets, i. e., the object category and boundaries, are generally different.
DIFER: Differentiable Automated Feature Engineering
Extensive experiments on classification and regression datasets demonstrate that DIFER can significantly improve the performance of various machine learning algorithms and outperform current state-of-the-art AutoFE methods in terms of both efficiency and performance.
Semi-supervised Embedding Learning for High-dimensional Bayesian Optimization
Bayesian optimization is a broadly applied methodology to optimize the expensive black-box function.
Object Relational Graph with Teacher-Recommended Learning for Video Captioning
In this paper, we propose a complete video captioning system including both a novel model and an effective training strategy.
Ranked #9 on
Video Captioning
on VATEX
(using extra training data)
VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning
Multi-modal information is essential to describe what has happened in a video.
Multimodal Semantic Attention Network for Video Captioning
Inspired by the fact that different modalities in videos carry complementary information, we propose a Multimodal Semantic Attention Network(MSAN), which is a new encoder-decoder framework incorporating multimodal semantic attributes for video captioning.
Interaction-aware Spatio-temporal Pyramid Attention Networks for Action Classification
Furthermore, since different layers in a deep network capture feature maps of different scales, we use these feature maps to construct a spatial pyramid and then utilize multi-scale information to obtain more accurate attention scores, which are used to weight the local features in all spatial positions of feature maps to calculate attention maps.
Spatio-Temporal Self-Organizing Map Deep Network for Dynamic Object Detection From Videos
In dynamic object detection, it is challenging to construct an effective model to sufficiently characterize the spatial-temporal properties of the background.
Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks
In this paper, we study the problem of online action detection from streaming skeleton data.
Multi-Feature Max-Margin Hierarchical Bayesian Model for Action Recognition
In this paper, a multi-feature max-margin hierarchical Bayesian model (M3HBM) is proposed for action recognition.
Multi-target Tracking with Motion Context in Tensor Power Iteration
In this paper, we model interactions between neighbor targets by pair-wise motion context, and further encode such context into the global association optimization.
Human Action Recognition Based on Context-Dependent Graph Kernels
Then, the proposed CGKs are applied to measure the similarity between actions represented by the two-graph model.
Multi-task Sparse Learning with Beta Process Prior for Action Recognition
In this paper, we formulate human action recognition as a novel Multi-Task Sparse Learning(MTSL) framework which aims to construct a test sample with multiple features from as few bases as possible.
3D R Transform on Spatio-temporal Interest Points for Action Recognition
In this paper, we propose a new global feature to capture the detailed geometrical distribution of interest points.
