Search Results for author:
Found 86 papers, 42 papers with code
Discovering Sounding Objects by Audio Queries for Audio Visual Segmentation
Explicit object-level semantic correspondence between audio and visual modalities is established by gathering object information from visual features with predefined audio queries.
ReSimAD: Zero-Shot 3D Domain Transfer for Autonomous Driving with Source Reconstruction and Target Simulation
Domain shifts such as sensor type changes and geographical situation variations are prevalent in Autonomous Driving (AD), which poses a challenge since AD model relying on the previous-domain knowledge can be hardly directly deployed to a new domain without additional costs.
Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception
Specifically, we analyze the performance changes of different methods under different bandwidths, providing a deep insight into the performance-bandwidth trade-off issue.
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
CCPD transfers the fundamental, point-to-point wayfinding skill that is well trained on the large-scale PointGoal task to ORAN, so as to help ORAN to better master audio-visual navigation with far fewer training samples.
DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
To address these limitations, we present a novel cascaded motion diffusion model, DiffDance, designed for high-resolution, long-form dance generation.
LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT
We introduce LyricWhiz, a robust, multilingual, and zero-shot automatic lyrics transcription method achieving state-of-the-art performance on various lyrics transcription datasets, even in challenging genres such as rock and metal.
MARBLE: Music Audio Representation Benchmark for Universal Evaluation
This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark.
A Tale of Two Approximations: Tightening Over-Approximation for DNN Robustness Verification via Under-Approximation
In this paper, we study existing approaches and identify a dominant factor in defining tight approximation, namely the approximation domain of the activation function.
Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels
Besides, the generated pseudo-labels can be fluctuating and inaccurate at the early stage of training.
Pseudo Label
Weakly-supervised Temporal Action Localization
+1
DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection
Second, through our design, the object queries and the foreground query in the decoder share consensus on the class semantics, therefore making the strong and weak supervision mutually benefit each other for domain alignment.
Sparse Dense Fusion for 3D Object Detection
With the prevalence of multimodal learning, camera-LiDAR fusion has gained popularity in 3D object detection.
Boosting Verified Training for Robust Image Classifications via Abstraction
Via abstraction, all perturbed images are mapped into intervals before feeding into neural networks for training.
Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection
When extracting object knowledge from PVLMs, the former adaptively transforms object proposals and adopts object-aware mask attention to obtain precise and complete knowledge of objects.
Ranked #7 on
Open Vocabulary Object Detection
on MSCOCO
FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR).
Anchor3DLane: Learning to Regress 3D Anchors for Monocular 3D Lane Detection
An attempt has been made to get rid of BEV and predict 3D lanes from FV representations directly, while it still underperforms other BEV-based methods given its lack of structured representation for 3D lanes.
Ranked #3 on
3D Lane Detection
on Apollo Synthetic 3D Lane
Object as Query: Lifting any 2D Object Detector to 3D Detection
For the generated queries, we design a sparse cross attention module to force them to focus on the features of specific objects, which suppresses interference from noises.
Adaptive Zone-Aware Hierarchical Planner for Vision-Language Navigation
In this paper, we propose an Adaptive Zone-aware Hierarchical Planner (AZHP) to explicitly divides the navigation process into two heterogeneous phases, i. e., sub-goal setting via zone partition/selection (high-level action) and sub-goal executing (low-level action), for hierarchical planning.
Bridging Search Region Interaction With Template for RGB-T Tracking
To alleviate these limitations, we propose a novel Template-Bridged Search region Interaction (TBSI) module which exploits templates as the medium to bridge the cross-modal interaction between RGB and TIR search regions by gathering and distributing target-relevant object and environment contexts.
Masked Contrastive Pre-Training for Efficient Video-Text Retrieval
Our MAC aims to reduce video representation's spatial and temporal redundancy in the VidLP model by a mask sampling mechanism to improve pre-training efficiency.
Ranked #31 on
Video Retrieval
on MSR-VTT-1kA
(using extra training data)
Analyzing Infrastructure LiDAR Placement with Realistic LiDAR Simulation Library
Through simulating point cloud data in different LiDAR placements, we can evaluate the perception accuracy of these placements using multiple detection models.
Teach-DETR: Better Training DETR with Teachers
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors.
BBReach: Tight and Scalable Black-Box Reachability Analysis of Deep Reinforcement Learning Systems
In this paper we propose a novel, tight and scalable reachability analysis approach for DRL systems.
Video Background Music Generation: Dataset, Method and Evaluation
We believe our dataset, benchmark model, and evaluation metric will boost the development of video background music generation.
DualApp: Tight Over-Approximation for Neural Network Robustness Verification via Under-Approximation
We observe that existing approaches only rely on overestimated domains, while the corresponding tight approximation may not necessarily be tight on its actual domain.
Cross-Modality Domain Adaptation for Freespace Detection: A Simple yet Effective Baseline
To better bridge the domain gap between source domain (synthetic data) and target domain (real-world data), we also propose a Selective Feature Alignment (SFA) module which only aligns the features of consistent foreground area between the two domains, thus realizing inter-domain intra-modality adaptation.
Multi-view Human Body Mesh Translator
Existing methods for human mesh recovery mainly focus on single-view frameworks, but they often fail to produce accurate results due to the ill-posed setup.
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance.
Provably Tightest Linear Approximation for Robustness Verification of Sigmoid-like Neural Networks
Considerable efforts have been devoted to finding the so-called tighter approximations to obtain more precise verification results.
PoseTrans: A Simple Yet Effective Pose Transformation Augmentation for Human Pose Estimation
Human pose estimation aims to accurately estimate a wide variety of human poses.
PPMN: Pixel-Phrase Matching Network for One-Stage Panoptic Narrative Grounding
To alleviate these drawbacks, we propose a one-stage end-to-end Pixel-Phrase Matching Network (PPMN), which directly matches each phrase to its corresponding pixels instead of region proposals and outputs panoptic segmentation by simple combination.
Target-Driven Structured Transformer Planner for Vision-Language Navigation
Vision-language navigation is the task of directing an embodied agent to navigate in 3D scenes with natural language instructions.
HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors
We observe that the core difficulty for heterogeneous KD (hetero-KD) is the significant semantic gap between the backbone features of heterogeneous detectors due to the different optimization manners.
Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation
Referring video object segmentation aims to predict foreground labels for objects referred by natural language expressions in videos.
Ranked #5 on
Referring Video Object Segmentation
on MeViS
Reinforced Structured State-Evolution for Vision-Language Navigation
However, the crucial navigation clues (i. e., object-level environment layout) for embodied navigation task is discarded since the maintained vector is essentially unstructured.
3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selection
3D visual grounding aims to locate the referred target object in 3D point cloud scenes according to a free-form language description.
TR-MOT: Multi-Object Tracking by Reference
RS takes previous detected results as references to aggregate the corresponding features from the combined features of the adjacent frames and makes a one-to-one track state prediction for each reference in parallel.
GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection
In this paper, we reveal and address the disadvantages of the conventional query-driven HOI detectors from the two aspects.
Ranked #11 on
Human-Object Interaction Detection
on HICO-DET
Distribution-Aware Single-Stage Models for Multi-Person 3D Pose Estimation
In this paper, we present a novel Distribution-Aware Single-stage (DAS) model for tackling the challenging multi-person 3D pose estimation problem.
3D Multi-Person Pose Estimation (absolute)
3D Multi-Person Pose Estimation (root-relative)
+2
Improved Pillar with Fine-grained Feature for 3D Object Detection
In contrast, the 2D grid-based methods, such as PointPillar, can easily achieve a stable and efficient speed based on simple 2D convolution, but it is hard to get the competitive accuracy limited by the coarse-grained point clouds representation.
Mining the Benefits of Two-stage and One-stage HOI Detection
To this end, we propose a novel one-stage framework with disentangling human-object detection and interaction classification in a cascade manner.
Ranked #7 on
Human-Object Interaction Detection
on V-COCO
TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding
Existing works usually adopt dynamic graph networks to indirectly model the intra/inter-modal interactions, making the model difficult to distinguish the referred object from distractors due to the monolithic representations of visual and linguistic contents.
Room-and-Object Aware Knowledge Reasoning for Remote Embodied Referring Expression
The Remote Embodied Referring Expression (REVERIE) is a recently raised task that requires an agent to navigate to and localise a referred remote object according to a high-level language instruction.
Discriminative Triad Matching and Reconstruction for Weakly Referring Expression Grounding
In this paper, we are tackling the weakly-supervised referring expression grounding task, for the localization of a referent object in an image according to a query sentence, where the mapping between image regions and queries are not available during the training stage.
PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively.
Human-centric Relation Segmentation: Dataset and Solution
For the above exemplar case, our HRS task produces results in the form of relation triplets <girl [left hand], hold, book> and exacts segmentation masks of the book, with which the robot can easily accomplish the grabbing task.
Cross-Modal Progressive Comprehension for Referring Segmentation
In this paper, we propose a Cross-Modal Progressive Comprehension (CMPC) scheme to effectively mimic human behaviors and implement it as a CMPC-I (Image) module and a CMPC-V (Video) module to improve referring image and video segmentation models.
Ranked #7 on
Referring Expression Segmentation
on J-HMDB
Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentation
Though 3D convolutions are amenable to recognizing which actor is performing the queried actions, it also inevitably introduces misaligned spatial information from adjacent frames, which confuses features of the target frame and yields inaccurate segmentation.
Ranked #8 on
Referring Expression Segmentation
on J-HMDB
Reformulating HOI Detection as Adaptive Set Prediction
To attain this, we map a trainable interaction query set to an interaction prediction set with a transformer.
Ranked #27 on
Human-Object Interaction Detection
on HICO-DET
(using extra training data)
Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing
To address the challenging task of instance-aware human part parsing, a new bottom-up regime is proposed to learn category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner.
General Instance Distillation for Object Detection
In recent years, knowledge distillation has been proved to be an effective solution for model compression.
Video Relation Detection with Trajectory-aware Multi-modal Features
Considering the complexity of doing visual relation detection in videos, we decompose this task into three sub-tasks: object detection, trajectory proposal and relation prediction.
ORDNet: Capturing Omni-Range Dependencies for Scene Parsing
Our ORDNet is able to extract more comprehensive context information and well adapt to complex spatial variance in scene images.
Language-Guided Global Image Editing via Cross-Modal Cyclic Mechanism
Given the cycle, we propose several free augmentation strategies to help our model understand various editing requests given the imbalanced dataset.
Confidence-aware Non-repetitive Multimodal Transformers for TextCaps
Our CNMT consists of a reading, a reasoning and a generation modules, in which Reading Module employs better OCR systems to enhance text reading ability and a confidence embedding to select the most noteworthy tokens.
Human-centric Spatio-Temporal Video Grounding With Visual Transformers
HC-STVG is a video grounding task that requires both spatial (where) and temporal (when) localization.
Linguistic Structure Guided Context Modeling for Referring Image Segmentation
Referring image segmentation aims to predict the foreground mask of the object referred by a natural language sentence.
Referring Image Segmentation via Cross-Modal Progressive Comprehension
In addition to the CMPC module, we further leverage a simple yet effective TGFE module to integrate the reasoned multimodal features from different levels with the guidance of textual information.
Ranked #11 on
Referring Expression Segmentation
on RefCOCO testB
Recapture as You Want
LGR module utilizes body skeleton knowledge to construct a layout graph that connects all relevant part features, where graph reasoning mechanism is used to propagate information among part nodes to mine their relations.
Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Video
Temporally language grounding in untrimmed videos is a newly-raised task in video understanding.
PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection
Human and object points are the center of the detection boxes, and the interaction point is the midpoint of the human and object points.
Ranked #24 on
Human-Object Interaction Detection
on V-COCO
AdversarialNAS: Adversarial Neural Architecture Search for GANs
In this paper, we propose an AdversarialNAS method specially tailored for Generative Adversarial Networks (GANs) to search for a superior generative model on the task of unconditional image generation.
Rule-Guided Compositional Representation Learning on Knowledge Graphs
Representation learning on a knowledge graph (KG) is to embed entities and relations of a KG into low-dimensional continuous vector spaces.
RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment
First, it can exploit pixel alignment and feature alignment jointly.
Cross-Modality Person Re-identification
Person Re-Identification
+1
Learning to Recognize the Unseen Visual Predicates
Visual relationship recognition models are limited in the ability to generalize from finite seen predicates to unseen ones.
PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer
In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image.
A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehension
RCCF reformulates the referring expression comprehension as a correlation filtering process.
UGAN: Untraceable GAN for Multi-Domain Face Translation
To address this issue, we propose a method called Untraceable GAN, which has a novel source classifier to differentiate which domain an image is translated from, and determines whether the translated image still retains the characteristics of the source domain.
Accurate facial image parsing at real-time speed
In this paper, we propose a design scheme for deep learning networks in the face parsing task with promising accuracy and real-time inference speed.
Ranked #6 on
Face Parsing
on CelebAMask-HQ
Open Category Detection with PAC Guarantees
Further, while there are algorithms for open category detection, there are few empirical results that directly report alien detection rates.
Ensemble Soft-Margin Softmax Loss for Image Classification
On the other hand, the learned classifier of softmax loss is weak.
Face Aging with Contextual Generative Adversarial Nets
The age discriminative network guides the synthesized face to fit the real conditional distribution.
Cross-domain Human Parsing via Adversarial Feature and Label Adaptation
Our proposed model explicitly learns a feature compensation network, which is specialized for mitigating the cross-domain differences.
Fast Deep Matting for Portrait Animation on Mobile Phone
Finally, an automatic portrait animation system based on fast deep matting is built on mobile devices, which does not need any interaction and can realize real-time matting with 15 fps.
Learning Adaptive Receptive Fields for Deep Image Parsing Network
In this paper, we introduce a novel approach to regulate receptive field in deep image parsing network automatically.
Surveillance Video Parsing with Single Frame Supervision
In this paper, we develop a Single frame Video Parsing (SVP) method which requires only one labeled frame per video in training stage.
SketchNet: Sketch Classification With Web Images
In this study, we present a weakly supervised approach that discovers the discriminative structures of sketch images, given pairs of sketch images and web images.
Structural Correlation Filter for Robust Visual Tracking
In this paper, we propose a novel structural correlation filter (SCF) model for robust visual tracking.
Makeup like a superstar: Deep Localized Makeup Transfer Network
In this paper, we propose a novel Deep Localized Makeup Transfer Network to automatically recommend the most suitable makeup for a female and synthesis the makeup on her face.
Low-Rank Tensor Constrained Multiview Subspace Clustering
We introduce a low-rank tensor constraint to explore the complementary information from multiple views and, accordingly, establish a novel method called Low-rank Tensor constrained Multiview Subspace Clustering (LT-MSC).
Towards Computational Baby Learning: A Weakly-Supervised Approach for Object Detection
Then the concept detector can be fine-tuned based on these new instances.
Human Parsing With Contextualized Convolutional Neural Network
In this work, we address the human parsing task with a novel Contextualized Convolutional Neural Network (Co-CNN) architecture, which well integrates the cross-layer context, global image-level context, within-super-pixel context and cross-super-pixel neighborhood context into a unified network.
Diversity-Induced Multi-View Subspace Clustering
In this paper, we focus on how to boost the multi-view clustering by exploring the complementary information among multi-view features.
Structural Sparse Tracking
Sparse representation has been applied to visual tracking by finding the best target candidate with minimal reconstruction error by use of target templates.
Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
Under the classic K Nearest Neighbor (KNN)-based nonparametric framework, the parametric Matching Convolutional Neural Network (M-CNN) is proposed to predict the matching confidence and displacements of the best matched region in the testing image for a particular semantic region in one KNN image.
Deep Human Parsing with Active Template Regression
The first CNN network is with max-pooling, and designed to predict the template coefficients for each label mask, while the second CNN network is without max-pooling to preserve sensitivity to label mask position and accurately predict the active shape parameters.
Computational Baby Learning
Then the concept detector can be fine-tuned based on these new instances.
