Search Results for author:
Found 54 papers, 25 papers with code
Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network
Then, the descriptions of the objects are served as a bridge to determine the importance of the association between the objects of image modality and the contextual words of text modality, so as to build a cross-modal graph for each multi-modal instance.
CPGAN: Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis
Typical methods for text-to-image synthesis seek to design effective generative architecture to model the text-to-image mapping directly.
UniVoxel: Fast Inverse Rendering by Unified Voxelization of Scene Representation
In this work we design a Unified Voxelization framework for explicit learning of scene representations, dubbed UniVoxel, which allows for efficient modeling of the geometry, materials and illumination jointly, thereby accelerating the inverse rendering significantly.
WeCromCL: Weakly Supervised Cross-Modality Contrastive Learning for Transcription-only Supervised Text Spotting
In this work, we formulate this challenging problem as a Weakly Supervised Cross-modality Contrastive Learning problem, and design a simple yet effective model dubbed WeCromCL that is able to detect each transcription in a scene image in a weakly supervised manner.
AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation
To address these challenges, we propose AnyControl, a multi-control image synthesis framework that supports arbitrary combinations of diverse control signals.
Domain-Rectifying Adapter for Cross-Domain Few-Shot Segmentation
Instead, our key idea is to adapt a small adapter for rectifying diverse target domain styles to the source domain.
Saliency-Aware Regularized Graph Neural Network
In this work, we propose the Saliency-Aware Regularized Graph Neural Network (SAR-GNN) for graph classification, which consists of two core modules: 1) a traditional graph neural network serving as the backbone for learning node features and 2) the Graph Neural Memory designed to distill a compact graph representation from node features of the backbone.
Robust 3D Tracking with Quality-Aware Shape Completion
Considering that, we propose to construct a synthetic target representation composed of dense and complete point clouds depicting the target shape precisely by shape completion for robust 3D tracking.
SA$^2$VP: Spatially Aligned-and-Adapted Visual Prompt
Typical methods for visual prompt tuning follow the sequential modeling paradigm stemming from NLP, which represents an input image as a flattened sequence of token embeddings and then learns a set of unordered parameterized tokens prefixed to the sequence representation as the visual prompts for task adaptation of large vision models.
D$^2$ST-Adapter: Disentangled-and-Deformable Spatio-Temporal Adapter for Few-shot Action Recognition
In particular, we devise the anisotropic Deformable Spatio-Temporal Attention module as the core component of D$^2$ST-Adapter, which can be tailored with anisotropic sampling densities along spatial and temporal domains to learn spatial and temporal features specifically in corresponding pathways, allowing our D$^2$ST-Adapter to encode features in a global view in 3D spatio-temporal space while maintaining a lightweight design.
Hierarchical Contrastive Learning for Pattern-Generalizable Image Corruption Detection
Detecting corrupted regions by learning the contrastive distinctions rather than the semantic patterns of corruptions, our model has well generalization ability across different corruption patterns.
Scene-Generalizable Interactive Segmentation of Radiance Fields
Existing methods for interactive segmentation in radiance fields entail scene-specific optimization and thus cannot generalize across different scenes, which greatly limits their applicability.
Feature Decoupling-Recycling Network for Fast Interactive Segmentation
First, our model decouples the learning of source image semantics from the encoding of user guidance to process two types of input domains separately.
Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement
Although extensive research has been conducted on 3D point cloud segmentation, effectively adapting generic models to novel categories remains a formidable challenge.
Reliability-Hierarchical Memory Network for Scribble-Supervised Video Object Segmentation
To be specific, RHMNet first only uses the memory in the high-reliability level to locate the region with high reliability belonging to the target, which is highly similar to the initial target scribble.
Audio2Gestures: Generating Diverse Gestures from Audio
Finally, we demonstrate that our method can be readily used to generate motion sequences with user-specified motion clips on the timeline.
Activating the Discriminability of Novel Classes for Few-shot Segmentation
To circumvent these two challenges, we propose to activate the discriminability of novel classes explicitly in both the feature encoding stage and the prediction stage for segmentation.
Semantic-Aware Local-Global Vision Transformer
Second, instead of only performing local self-attention within local windows as Swin Transformer does, the proposed SALG performs both 1) local intra-region self-attention for learning fine-grained features within each region and 2) global inter-region feature propagation for modeling global dependencies among all regions.
Ranked #930 on
Image Classification
on ImageNet
Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid
In this paper, we find that this problem usually occurs when the positions of support samples are in the vicinity of task centroid -- the mean of all class centroids in the task.
SSORN: Self-Supervised Outlier Removal Network for Robust Homography Estimation
In this paper, we attempt to build a deep learning model that mimics all four steps in the traditional homography estimation pipeline.
Layout-Bridging Text-to-Image Synthesis
Specifically, we formulate the text-to-layout generation as a sequence-to-sequence modeling task, and build our model upon Transformer to learn the spatial relationships between objects by modeling the sequential dependencies between them.
Learning Generalizable Latent Representations for Novel Degradations in Super Resolution
The real-world degradations can be beyond the simulation scope by the handcrafted degradations, which are referred to as novel degradations.
Few-Shot Object Detection by Knowledge Distillation Using Bag-of-Visual-Words Representations
In this work we design a novel knowledge distillation framework to guide the learning of the object detector and thereby restrain the overfitting in both the pre-training stage on base classes and fine-tuning stage on novel classes.
Self-Support Few-Shot Semantic Segmentation
Motivated by the simple Gestalt principle that pixels belonging to the same object are more similar than those to different objects of same class, we propose a novel self-support matching strategy to alleviate this problem, which uses query prototypes to match query features, where the query prototypes are collected from high-confidence query predictions.
Ranked #13 on
Few-Shot Semantic Segmentation
on FSS-1000 (5-shot)
Multi-Faceted Distillation of Base-Novel Commonality for Few-shot Object Detection
Most of existing methods for few-shot object detection follow the fine-tuning paradigm, which potentially assumes that the class-agnostic generalizable knowledge can be learned and transferred implicitly from base classes with abundant samples to novel classes with limited samples via such a two-stage training strategy.
Learning Sequence Representations by Non-local Recurrent Neural Memory
The key challenge of sequence representation learning is to capture the long-range temporal dependencies.
Global-Local Stepwise Generative Network for Ultra High-Resolution Image Restoration
While the research on image background restoration from regular size of degraded images has achieved remarkable progress, restoring ultra high-resolution (e. g., 4K) images remains an extremely challenging task due to the explosion of computational complexity and memory usage, as well as the deficiency of annotated data.
Single Shot Self-Reliant Scene Text Spotter by Decoupled yet Collaborative Detection and Recognition
Typical text spotters follow the two-stage spotting paradigm which detects the boundary for a text instance first and then performs text recognition within the detected regions.
Ranked #5 on
Text Spotting
on ICDAR 2015
Global Tracking via Ensemble of Local Trackers
While it can exploit the temporal context like historical appearances and locations of the target, a potential limitation of such strategy is that the local tracker tends to misidentify a nearby distractor as the target instead of activating the re-detector when the real target is out of view.
Exploring Category-correlated Feature for Few-shot Image Classification
To alleviate this problem, we present a simple yet effective feature rectification method by exploring the category correlation between novel and base classes as the prior knowledge.
An Informative Tracking Benchmark
Along with the rapid progress of visual tracking, existing benchmarks become less informative due to redundancy of samples and weak discrimination between current trackers, making evaluations on all datasets extremely time-consuming.
U2-Former: A Nested U-shaped Transformer for Image Restoration
While Transformer has achieved remarkable performance in various high-level vision tasks, it is still challenging to exploit the full potential of Transformer in image restoration.
Ranked #21 on
Image Dehazing
on SOTS Indoor
Pedestrian Detection by Exemplar-Guided Contrastive Learning
Instead of learning each of these diverse pedestrian appearance features individually as most existing methods do, we propose to perform contrastive learning to guide the feature learning in such a way that the semantic distance between pedestrians with different appearances in the learned feature space is minimized to eliminate the appearance diversities, whilst the distance between pedestrians and background is maximized.
Ranked #1 on
Pedestrian Detection
on TJU-Ped-campus
Label-Aware Distribution Calibration for Long-tailed Classification
Real-world data usually present long-tailed distributions.
Stepwise-Refining Speech Separation Network via Fine-Grained Encoding in High-order Latent Domain
Existing methods for speech separation either transform the speech signals into frequency domain to perform separation or seek to learn a separable embedding space by constructing a latent domain based on convolutional filters.
Ranked #10 on
Speech Separation
on WHAMR!
Generative Memory-Guided Semantic Reasoning Model for Image Inpainting
Most existing methods for image inpainting focus on learning the intra-image priors from the known regions of the current input image to infer the content of the corrupted regions in the same image.
Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders
In order to overcome this problem, we propose a novel conditional variational autoencoder (VAE) that explicitly models one-to-many audio-to-motion mapping by splitting the cross-modal latent code into shared code and motion-specific code.
Ranked #3 on
Gesture Generation
on BEAT
Saliency-Associated Object Tracking
A potential limitation of such trackers is that not all patches are equally informative for tracking.
Self-Supervised Tracking via Target-Aware Data Synthesis
While deep-learning based tracking methods have achieved substantial progress, they entail large-scale and high-quality annotated data for sufficient training.
SiamCorners: Siamese Corner Networks for Visual Tracking
By tracking a target as a pair of corners, we avoid the need to design the anchor boxes.
Deep-Masking Generative Network: A Unified Framework for Background Restoration from Superimposed Images
In this work we present the Deep-Masking Generative Network (DMGN), which is a unified framework for background restoration from the superimposed images and is able to cope with different types of noise.
Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation
We propose to learn the underlying class-agnostic commonalities that can be generalized from mask-annotated categories to novel categories.
Ranked #81 on
Instance Segmentation
on COCO test-dev
CPGAN: Full-Spectrum Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis
Typical methods for text-to-image synthesis seek to design effective generative architecture to model the text-to-image mapping directly.
Push for Quantization: Deep Fisher Hashing
In this paper we push for quantization: We optimize maximum class separability in the binary space.
Reflective Decoding Network for Image Captioning
State-of-the-art image captioning methods mostly focus on improving visual features, less attention has been paid to utilizing the inherent properties of language to boost captioning performance.
Ranked #5 on
Image Captioning
on MS COCO
Push for Center Learning via Orthogonalization and Subspace Masking for Person Re-Identification
Person re-identification aims to identify whether pairs of images belong to the same person or not.
Non-local Recurrent Neural Memory for Supervised Sequence Modeling
Typical methods for supervised sequence modeling are built upon the recurrent neural networks to capture temporal dependencies.
Memory-Attended Recurrent Network for Video Captioning
Typical techniques for video captioning follow the encoder-decoder framework, which can only focus on one source video being processed.
Unsupervised Learning of Sequence Representations by Autoencoders
Sequence data is challenging for machine learning approaches, because the lengths of the sequences may vary between samples.
Attended End-to-end Architecture for Age Estimation from Facial Expression Videos
In this paper, we present an end-to-end architecture for age estimation, called Spatially-Indexed Attention Model (SIAM), which is able to simultaneously learn both the appearance and dynamics of age from raw videos of facial expressions.
Interacting Attention-gated Recurrent Networks for Recommendation
In particular, we propose a novel attention scheme to learn the attention scores of user and item history in an interacting way, thus to account for the dependencies between user and item dynamics in shaping user-item interactions.
Temporal Attention-Gated Model for Robust Sequence Classification
An important advantage of our approach is interpretability since the temporal attention weights provide a meaningful value for the salience of each time step in the sequence.
Modeling Time Series Similarity with Siamese Recurrent Networks
Traditional techniques for measuring similarities between time series are based on handcrafted similarity measures, whereas more recent learning-based approaches cannot exploit external supervision.
Time Series Classification using the Hidden-Unit Logistic Model
We present a new model for time series classification, called the hidden-unit logistic model, that uses binary stochastic hidden units to model latent structure in the data.
