Search Results for author:
Found 38 papers, 25 papers with code
Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network
Given some video-query pairs with untrimmed videos and sentence queries, temporal sentence grounding (TSG) aims to locate query-relevant segments in these videos.
Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval
As a general parameter-efficient way, a common solution is to utilize adapter modules to transfer the vision-language alignment ability of Vision-Language Pretraining (VLP) models from a source language to a target language.
Rethinking Video Deblurring with Wavelet-Aware Dynamic Transformer and Diffusion Model
Since Diffusion Models (DMs) have strong capabilities in generating high-frequency details, we consider introducing DMs into the video deblurring task.
Mitigating Multilingual Hallucination in Large Vision-Language Models
In this paper, we make the first attempt to mitigate this important multilingual hallucination in LVLMs.
Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation
To assess the capability of our proposed ARA model in reducing hallucination, we employ three widely used LVLM models (LLaVA-1. 5, Qwen-VL, and mPLUG-Owl2) across four benchmarks.
Representation Alignment Contrastive Regularization for Multi-Object Tracking
Achieving high-performance in multi-object tracking algorithms heavily relies on modeling spatio-temporal relationships during the data association stage.
Let All be Whitened: Multi-teacher Distillation for Efficient Visual Retrieval
Instead of crafting a new method pursuing further improvement on accuracy, in this paper we propose a multi-teacher distillation framework Whiten-MTD, which is able to transfer knowledge from off-the-shelf pre-trained retrieval models to a lightweight student model for efficient visual retrieval.
CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer
Cross-lingual cross-modal retrieval has garnered increasing attention recently, which aims to achieve the alignment between vision and target language (V-T) without using any annotated V-T data pairs.
Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding
In this manner, our framework is able to learn the unified representations of uni-modal or multi-modal skeleton input, which is flexible to different kinds of modality input for robust action understanding in practical cases.
Video Infringement Detection via Feature Disentanglement and Mutual Information Maximization
We theoretically analyzed the mutual information between the label and the disentangled features, arriving at a loss that maximizes the extraction of task-relevant information from the original feature.
Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrieval
Therefore, we propose Dual-view Curricular Optimal Transport (DCOT) to learn with noisy correspondence in CCR.
From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval
As the attribute-specific similarity typically corresponds to the specific subtle regions of images, we propose a Region-to-Patch Framework (RPF) that consists of a region-aware branch and a patch-aware branch to extract fine-grained attribute-related visual features for precise retrieval in a coarse-to-fine manner.
Transform-Equivariant Consistency Learning for Temporal Sentence Grounding
This paper addresses the temporal sentence grounding (TSG).
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval
During the knowledge distillation, an inheritance student branch is devised to absorb the knowledge from the teacher model.
Hierarchical Contrast for Unsupervised Skeleton-based Action Representation Learning
This paper targets unsupervised skeleton-based action representation learning and proposes a new Hierarchical Contrast (HiCo) framework.
Action Recognition
Few-Shot Skeleton-Based Action Recognition
+4
Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning
In this paper, we propose a noise-robust cross-lingual cross-modal retrieval method for low-resource languages.
Partially Relevant Video Retrieval
To fill the gap, we propose in this paper a novel T2VR subtask termed Partially Relevant Video Retrieval (PRVR).
Ranked #1 on
Partially Relevant Video Retrieval
on TVR
Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval
In this work, we concentrate on video representation learning, an essential component for text-to-video retrieval.
Lightweight Attentional Feature Fusion: A New Baseline for Text-to-Video Retrieval
In this paper we revisit feature fusion, an old-fashioned topic, in the new context of text-to-video retrieval.
Ranked #1 on
Ad-hoc video search
on TRECVID-AVS20 (V3C1)
(using extra training data)
Adaptive Proposal Generation Network for Temporal Sentence Localization in Videos
However, the performance of bottom-up model is inferior to the top-down counterpart as it fails to exploit the segment-level interaction.
Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning
In this similarity paradigm, one should pay more attention to the similarity in terms of a specific design/attribute between fashion items.
Context-aware Biaffine Localizing Network for Temporal Sentence Grounding
This paper addresses the problem of temporal sentence grounding (TSG), which aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query.
Hierarchical Similarity Learning for Language-based Product Image Retrieval
In this paper, we focus on the cross-modal similarity measurement, and propose a novel Hierarchical Similarity Learning (HSL) network.
Progressive Localization Networks for Language-based Moment Localization
The language-based setting of this task allows for an open set of target activities, resulting in a large variation of the temporal lengths of video moments.
Reasoning Step-by-Step: Temporal Sentence Localization in Videos via Deep Rectification-Modulation Network
In this paper, we propose a novel deep rectification-modulation network (RMN), transforming this task into a multi-step reasoning process by repeating rectification and modulation.
Dual Encoding for Video Retrieval by Text
In this paper we achieve this by proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own.
Ranked #3 on
Ad-hoc video search
on TRECVID-AVS16 (IACC.3)
(using extra training data)
Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videos
In this paper, we propose a Fine-grained Iterative Attention Network (FIAN) that consists of an iterative attention module for bilateral query-video in-formation extraction.
Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization
To this end, we propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph.
Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval
To facilitate video retrieval with complex queries, we propose a Tree-augmented Cross-modal Encoding method by jointly learning the linguistic structure of queries and the temporal representation of videos.
Feature Re-Learning with Data Augmentation for Video Relevance Prediction
Predicting the relevance between two given videos with respect to their visual content is a key component for content-based video recommendation and retrieval.
Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network
This paper strives to learn fine-grained fashion similarity.
Dual Encoding for Zero-Example Video Retrieval
This paper attacks the challenging problem of zero-example video retrieval.
Exploring Human-like Attention Supervision in Visual Question Answering
Since there is a lack of human attention data, we first propose a Human Attention Network (HAN) to generate human-like attention maps, training on a recently released dataset called Human ATtention Dataset (VQA-HAT).
Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild
To quantify the current progress, we propose a simple text2image method, representing a novel test query by a set of images selected from large-scale query log.
Fluency-Guided Cross-Lingual Image Captioning
The framework comprises a module to automatically estimate the fluency of the sentences and another module to utilize the estimated fluency scores to effectively train an image captioning model for the target language.
Learning Deep Representations Using Convolutional Auto-encoders with Symmetric Skip Connections
In this paper, we investigate convolutional denoising auto-encoders to show that unsupervised pre-training can still improve the performance of high-level image related tasks such as image classification and semantic segmentation.
Word2VisualVec: Image and Video to Sentence Matching by Visual Feature Prediction
This paper strives to find the sentence best describing the content of an image or video.
