Search Results for author:
Found 39 papers, 11 papers with code
Learning to Focus on the Foreground for Temporal Sentence Grounding
Then, we develop a self-supervised coarse-to-fine paradigm to learn to locate the most query-relevant patch in each frame and aggregate them among the video for final grounding.
A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
Compared to traditional Large Language Models (LLMs), LVLMs present great potential and challenges due to its closer proximity to the multi-resource real-world applications and the complexity of multi-modal processing.
A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions
In this survey, we attempt to provide a comprehensive overview of the T-3DVG progress, including its fundamental elements, recent research advances, and future research directions.
Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding
In this manner, our framework is able to learn the unified representations of uni-modal or multi-modal skeleton input, which is flexible to different kinds of modality input for robust action understanding in practical cases.
Dense Object Grounding in 3D Scenes
Localizing objects in 3D scenes according to the semantics of a given natural language is a fundamental yet important task in the field of multimedia understanding, which benefits various real-world applications such as robotics and autonomous driving.
3DHacker: Spectrum-based Decision Boundary Generation for Hard-label 3D Point Cloud Attack
With the maturity of depth sensors, the vulnerability of 3D point cloud models has received increasing attention in various applications such as autonomous driving and robot navigation.
From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval
As the attribute-specific similarity typically corresponds to the specific subtle regions of images, we propose a Region-to-Patch Framework (RPF) that consists of a region-aware branch and a patch-aware branch to extract fine-grained attribute-related visual features for precise retrieval in a coarse-to-fine manner.
Transform-Equivariant Consistency Learning for Temporal Sentence Grounding
This paper addresses the temporal sentence grounding (TSG).
Density-Insensitive Unsupervised Domain Adaption on 3D Object Detection
Recently, few works attempt to tackle the domain gap in objects, but still fail to adapt to the gap of varying beam-densities between two domains, which is critical to mitigate the characteristic differences of the LiDAR collectors.
You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos
To handle the raw video bit-stream input, we propose a novel Three-branch Compressed-domain Spatial-temporal Fusion (TCSF) framework, which extracts and aggregates three kinds of low-level visual features (I-frame, motion vector and residual features) for effective and efficient grounding.
Jointly Visual- and Semantic-Aware Graph Memory Networks for Temporal Sentence Localization in Videos
Temporal sentence localization in videos (TSLV) aims to retrieve the most interested segment in an untrimmed video according to a given sentence query.
Tracking Objects and Activities with Attention for Temporal Sentence Grounding
Temporal sentence grounding (TSG) aims to localize the temporal segment which is semantically aligned with a natural language query in an untrimmed video. Most existing methods extract frame-grained features or object-grained features by 3D ConvNet or detection network under a conventional TSG framework, failing to capture the subtle differences between frames or to model the spatio-temporal behavior of core persons/objects.
Hypotheses Tree Building for One-Shot Temporal Sentence Localization
Given an untrimmed video, temporal sentence localization (TSL) aims to localize a specific segment according to a given sentence query.
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding
All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning.
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval
During the knowledge distillation, an inheritance student branch is devised to absorb the knowledge from the teacher model.
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble
Distantly-Supervised Named Entity Recognition (DS-NER) effectively alleviates the data scarcity problem in NER by automatically generating training samples.
Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval
In addition, due to the domain gap between different datasets, directly applying these pre-trained models to an unseen domain leads to a significant performance drop.
Hierarchical Local-Global Transformer for Temporal Sentence Grounding
To address this issue, in this paper, we propose a novel Hierarchical Local-Global Transformer (HLGT) to leverage this hierarchy information and model the interactions between different levels of granularity and different modalities for learning more fine-grained multi-modal representations.
Point Cloud Attacks in Graph Spectral Domain: When 3D Geometry Meets Graph Signal Processing
Instead, we propose point cloud attacks from a new perspective -- the graph spectral domain attack, aiming to perturb graph transform coefficients in the spectral domain that corresponds to varying certain geometric structure.
Skimming, Locating, then Perusing: A Human-Like Framework for Natural Language Video Localization
SLP consists of a Skimming-and-Locating (SL) module and a Bi-directional Perusing (BP) module.
Reducing the Vision and Language Bias for Temporal Sentence Grounding
In this paper, we study the above issue of selection biases and accordingly propose a Debiasing-TSG (D-TSG) model to filter and remove the negative biases in both vision and language modalities for enhancing the model generalization ability.
Gaussian Kernel-based Cross Modal Network for Spatio-Temporal Video Grounding
Spatial-Temporal Video Grounding (STVG) is a challenging task which aims to localize the spatio-temporal tube of the interested object semantically according to a natural language query.
Multi-Scale Self-Contrastive Learning with Hard Negative Mining for Weakly-Supervised Query-based Video Grounding
Secondly, since some predicted frames (i. e., boundary frames) are relatively coarse and exhibit similar appearance to their adjacent frames, we propose a coarse-to-fine contrastive learning paradigm to learn more discriminative frame-wise representations for distinguishing the false positive frames.
Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Grounding
Temporal sentence grounding aims to localize a target segment in an untrimmed video semantically according to a given sentence query.
Exploring the Devil in Graph Spectral Domain for 3D Point Cloud Attacks
Instead, we propose point cloud attacks from a new perspective -- Graph Spectral Domain Attack (GSDA), aiming to perturb transform coefficients in the graph spectral domain that corresponds to varying certain geometric structure.
Unsupervised Temporal Video Grounding with Deep Semantic Clustering
Temporal video grounding (TVG) aims to localize a target segment in a video according to a given sentence query.
Exploring Motion and Appearance Information for Temporal Sentence Grounding
Then, we develop separate motion and appearance branches to learn motion-guided and appearance-guided object relations, respectively.
Memory-Guided Semantic Learning Network for Temporal Sentence Grounding
To tackle this issue, we propose a memory-augmented network, called Memory-Guided Semantic Learning Network (MGSL-Net), that learns and memorizes the rarely appeared content in TSG tasks.
Imperceptible Transfer Attack and Defense on 3D Point Cloud Classification
Although many efforts have been made into attack and defense on the 2D image domain in recent years, few methods explore the vulnerability of 3D models.
Adaptive Proposal Generation Network for Temporal Sentence Localization in Videos
However, the performance of bottom-up model is inferior to the top-down counterpart as it fails to exploit the segment-level interaction.
Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding
A key solution to temporal sentence grounding (TSG) exists in how to learn effective alignment between vision and language features extracted from an untrimmed video and a sentence description.
Context-aware Biaffine Localizing Network for Temporal Sentence Grounding
This paper addresses the problem of temporal sentence grounding (TSG), which aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query.
Spatiotemporal Graph Neural Network based Mask Reconstruction for Video Object Segmentation
To capture temporal information from previous frames, we use a memory network to refine the mask of current frame by retrieving historic masks in a temporal graph.
F2Net: Learning to Focus on the Foreground for Unsupervised Video Object Segmentation
Specifically, our proposed network consists of three main parts: Siamese Encoder Module, Center Guiding Appearance Diffusion Module, and Dynamic Information Fusion Module.
Ranked #10 on
Unsupervised Video Object Segmentation
on FBMS test
Semantic Segmentation
Unsupervised Video Object Segmentation
+1
Reasoning Step-by-Step: Temporal Sentence Localization in Videos via Deep Rectification-Modulation Network
In this paper, we propose a novel deep rectification-modulation network (RMN), transforming this task into a multi-step reasoning process by repeating rectification and modulation.
Video-based Facial Expression Recognition using Graph Convolutional Networks
In terms of video based FER task, it is sensible to capture the dynamic expression variation among the frames to recognize facial expression.
Facial Expression Recognition
Facial Expression Recognition (FER)
Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization
To this end, we propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph.
Dynamic Graph Correlation Learning for Disease Diagnosis with Incomplete Labels
In this work, we propose a Disease Diagnosis Graph Convolutional Network (DD-GCN) that presents a novel view of investigating the inter-dependency among different diseases by using a dynamic learnable adjacency matrix in graph structure to improve the diagnosis accuracy.
MHP-VOS: Multiple Hypotheses Propagation for Video Object Segmentation
Extensive experiments on challenging datasets demonstrate the effectiveness of the proposed method, especially in the case of object missing.
