Search Results for author:
Found 94 papers, 49 papers with code
3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for 3D Object Detection
Neural Radiance Fields (NeRF) are widely used for novel-view synthesis and have been adapted for 3D Object Detection (3DOD), offering a promising approach to 3DOD through view-synthesis representation.
DreamHead: Learning Spatial-Temporal Correspondence via Hierarchical Diffusion for Audio-driven Talking Head Synthesis
Audio-driven talking head synthesis strives to generate lifelike video portraits from provided audio.
Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling
To target this problem, we propose a novel approach of modeling non-rigid motions as radiance residual fields to benefit from more direct color supervision in the rendering and utilize the rigid radiance fields as a prior to reduce the complexity of the learning process.
Learning Online Scale Transformation for Talking Head Video Generation
To this end, we introduce a scale transformation module that can automatically adjust the scale of the driving image to fit that of the source image, by using the information of scale difference maintained in the detected keypoints of the source image and the driving frame.
Sample-efficient Imitative Multi-token Decision Transformer for Generalizable Real World Driving
Reinforcement learning via sequence modeling has shown remarkable promise in autonomous systems, harnessing the power of offline datasets to make informed decisions in simulated environments.
Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space
In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space.
RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting
In the coarse stage, RoomTex first unwraps the scene mesh to a panoramic depth map and leverages ControlNet to generate a room panorama, which is regarded as the coarse reference to ensure the global texture consistency.
Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detection
3D-NOD is further extended with an Enrichment strategy that significantly enriches the novel object distribution in the training scenes, and then enhances the model's ability to localize more novel objects.
X-VILA: Cross-Modality Alignment for Large Language Model
We introduce X-VILA, an omni-modality model designed to extend the capabilities of large language models (LLMs) by incorporating image, video, and audio modalities.
PyGS: Large-scale Scene Representation with Pyramidal 3D Gaussian Splatting
To address these challenges, we present Pyramidal 3D Gaussian Splatting (PyGS) with NeRF Initialization.
Efficient Multitask Dense Predictor via Binarization
Moreover, we introduce a knowledge distillation mechanism to correct the direction of information flow in backward propagation.
Interactive3D: Create What You Want by Interactive 3D Generation
Both of them limit customization to the confines of the 2D reference and potentially introduce undesirable artifacts during the 3D lifting process, restricting the scope for direct and versatile 3D modifications.
GScream: Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal
This paper tackles the intricate challenge of object removal to update the radiance field using the 3D Gaussian Splatting.
DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection
This is followed by a fine-tuning stage that leverages a small number of high-resolution samples to further enhance detection performance.
Ranked #2 on
Object Detection
on ODinW Full-Shot 13 Tasks
CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs
While existing works mainly consider ray-level consistency to construct 2D learning regularization based on rendered color, depth, or semantics on image planes, in this paper we propose a novel approach that models 3D spatial field consistency to improve NeRF's performance with sparse inputs.
DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Data
It designs a joint diffusion and denoising paradigm to model a potential noisy distribution in the task prediction or feature maps and generate rectified outputs for different tasks.
Personalized LoRA for Human-Centered Text Understanding
Effectively and efficiently adapting a pre-trained language model (PLM) for human-centered text understanding (HCTU) is challenging since user tokens are million-level in most personalized applications and do not have concrete explicit semantics.
Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation
We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from these saliency maps and the significant inter-task correlation between saliency detection and semantic segmentation.
Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors
Recently, researchers have attempted to improve the genuineness of 3D objects by directly training on 3D datasets, albeit at the cost of low-quality texture generation due to the limited texture diversity in 3D datasets.
GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
This strategy is essential to extend 3D Gaussian representation to reconstruct the whole scene rather than synthesize a static object in existing methods.
Implicit Event-RGBD Neural SLAM
To address these challenges, we propose EN-SLAM, the first event-RGBD implicit neural SLAM framework, which effectively leverages the high rate and high dynamic range advantages of event data for tracking and mapping.
SegGen: Supercharging Segmentation Models with Text2Mask and Mask2Img Synthesis
On the highly competitive ADE20K and COCO benchmarks, our data generation method markedly improves the performance of state-of-the-art segmentation models in semantic segmentation, panoptic segmentation, and instance segmentation.
CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detection
Open-vocabulary 3D Object Detection (OV-3DDet) aims to detect objects from an arbitrary list of categories within a 3D scene, which remains seldom explored in the literature.
MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation
Building upon the observation that the attended regions of the one-class token in the standard vision transformer can contribute to a class-agnostic localization map, we explore the potential of the transformer model to capture class-specific attention for class-discriminative object localization by learning multiple class tokens.
Object Localization
Weakly supervised Semantic Segmentation
+1
Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis
State-of-the-art methods in this direction typically consider building separate networks for these two tasks (i. e., view synthesis and editing).
TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Experts
Furthermore, to establish long-range modeling of the task-specific representations from different layers of TaskExpert, we design a multi-task feature memory that updates at each layer and acts as an additional feature expert for dynamic task-specific feature decoding.
Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation
Talking head video generation aims to animate a human face in a still image with dynamic poses and expressions using motion information derived from a target-driving video, while maintaining the person's identity in the source image.
Contrastive Multi-Task Dense Prediction
A core objective in design is how to effectively model cross-task interactions to achieve a comprehensive improvement on different tasks based on their inherent complementarity and consistency.
InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding
And then, we design a transformer decoder to establish spatial and cross-task interaction globally, and a novel UP-Transformer block is devised to increase the resolutions of multi-task features gradually and establish cross-task interaction at different scales.
DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generation
In this work, firstly, we present a novel self-supervised method for learning dense 3D facial geometry (ie, depth) from face videos, without requiring camera parameters and 3D geometry annotations in training.
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment
This paper presents DetCLIPv2, an efficient and scalable training framework that incorporates large-scale image-text pairs to achieve open-vocabulary object detection (OVD).
Ranked #5 on
Object Detection
on ODinW Full-Shot 13 Tasks
Joint 2D-3D Multi-Task Learning on Cityscapes-3D: 3D Detection, Segmentation, and Depth Estimation
TaskPrompter introduces a new multi-task benchmark based on Cityscapes-3D dataset, which requires the multi-task model to concurrently generate predictions for monocular 3D vehicle detection, semantic segmentation, and monocular depth estimation.
Ranked #1 on
Monocular Depth Estimation
on Cityscapes 3D
You Only Train Once: Multi-Identity Free-Viewpoint Neural Human Rendering from Monocular Videos
In this paper, we tackle this problem by proposing a set of learnable identity codes to expand the capability of the framework for multi-identity free-viewpoint rendering, and an effective pose-conditioned code query mechanism to finely model the pose-dependent non-rigid motions.
Learning Multi-Modal Class-Specific Tokens for Weakly Supervised Dense Object Localization
Weakly supervised dense object localization (WSDOL) relies generally on Class Activation Mapping (CAM), which exploits the correlation between the class weights of the image classifier and the pixel-level features.
LP-BFGS attack: An adversarial attack based on the Hessian with limited pixels
We study the attack performance and computation cost of the attack method based on the Hessian with a limited number of perturbation pixels.
DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection
We further design a concept dictionary~(with descriptions) from various online sources and detection datasets to provide prior knowledge for each concept.
Lipschitz Continuity Retained Binary Neural Network
Relying on the premise that the performance of a binary neural network can be largely restored with eliminated quantization error between full-precision weight vectors and their corresponding binary vectors, existing works of network binarization frequently adopt the idea of model robustness to reach the aforementioned objective.
Exploring Adversarial Examples and Adversarial Robustness of Convolutional Neural Networks by Mutual Information
Adversarial training is a simple and effective defense method to improve the robustness of CNNs to adversarial examples.
Network Binarization via Contrastive Learning
Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit.
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation
This problem has been widely investigated in the research community and several Incremental Learning (IL) approaches have been proposed in the past years.
An Adaptive and Scalable ANN-based Model-Order-Reduction Method for Large-Scale TO Designs
Most models are trained to work with the design problem similar to that used for data generation and require retraining if the design problem changes.
InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understanding
Multi-task dense scene understanding is a thriving research domain that requires simultaneous perception and reasoning on a series of correlated tasks with pixel-wise prediction.
Ranked #1 on
Boundary Detection
on NYU-Depth V2
Depth-Aware Generative Adversarial Network for Talking Head Video Generation
In a more dense way, the depth is also utilized to learn 3D-aware cross-modal (i. e. appearance and depth) attention to guide the generation of motion fields for warping source image representations.
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation
To this end, we propose a Multi-class Token Transformer, termed as MCTformer, which uses multiple class tokens to learn interactions between the class tokens and the patch tokens.
Continual Attentive Fusion for Incremental Learning in Semantic Segmentation
To fill this gap, in this paper we introduce a novel attentive feature distillation approach to mitigate catastrophic forgetting while accounting for semantic spatial- and channel-level dependencies.
Generalized Binary Search Network for Highly-Efficient Multi-View Stereo
The new formulation makes our method only sample a very small number of depth hypotheses in each step, which is highly memory efficient, and also greatly facilitates quick training convergence.
Ranked #8 on
3D Reconstruction
on DTU
Contrastive Mutual Information Maximization for Binary Neural Networks
Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit.
Fine-grained Domain Adaptive Crowd Counting via Point-derived Segmentation
To this end, we propose to untangle \emph{domain-invariant} crowd and \emph{domain-specific} background from crowd images and design a fine-grained domain adaption method for crowd counting.
Reducing Spatial Labeling Redundancy for Semi-supervised Crowd Counting
In this way, we can transfer the original spatial labeling redundancy caused by individual similarities to effective supervision signals on the unlabeled regions.
Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection
In this paper, we propose to learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection.
Ranked #10 on
Monocular 3D Object Detection
on KITTI Cars Moderate
Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization
In this work, we argue that the features extracted from the pretrained extractor, e. g., I3D, are not the WS-TALtask-specific features, thus the feature re-calibration is needed for reducing the task-irrelevant information redundancy.
Weakly Supervised Action Localization
Weakly-supervised Temporal Action Localization
+1
Cross-modal Consensus Network forWeakly Supervised Temporal Action Localization
In this work, we argue that the features extracted from the pretrained extractor, e. g., I3D, are not the WS-TALtask-specific features, thus the feature re-calibration is needed for reducing the task-irrelevant information redundancy.
Weakly-supervised Temporal Action Localization
Weakly Supervised Temporal Action Localization
Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation
Motivated by the significant inter-task correlation, we propose a novel weakly supervised multi-task framework termed as AuxSegNet, to leverage saliency detection and multi-label image classification as auxiliary tasks to improve the primary task of semantic segmentation using only image-level ground-truth labels.
SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks
To this end, we propose to learn implicit surface reconstruction by sign-agnostic optimization of convolutional occupancy networks, to simultaneously achieve advanced scalability to large-scale scenes, generality to novel shapes, and applicability to raw scans in a unified framework.
Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes
We build on the idea of view synthesis, which uses classical camera geometry to re-render a source image from a different point-of-view, specified by a predicted relative pose and depth map.
Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction
This paper focuses on the task of 4D shape reconstruction from a sequence of point clouds.
Delving into Localization Errors for Monocular 3D Object Detection
Estimating 3D bounding boxes from monocular images is an essential component in autonomous driving, while accurate 3D object detection from this kind of data is very challenging.
3D Object Detection From Monocular Images
Autonomous Driving
+3
Variational Structured Attention Networks for Deep Visual Representation Learning
Specifically, we integrate the estimation and the interaction of the attentions within a probabilistic representation learning framework, leading to Variational STructured Attention networks (VISTA-Net).
NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge.
Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction
In contrast to previous works directly considering multi-scale feature maps obtained from the inner layers of a primary CNN architecture, and simply fusing the features with weighted averaging or concatenation, we propose a probabilistic graph attention network structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs) model for learning and fusing multi-scale representations in a principled manner.
Variational Structured Attention Networks for Dense Pixel-Wise Prediction
State-of-the-art performances in dense pixel-wise prediction tasks are obtained with specifically designed convolutional networks.
Unsupervised Image Segmentation using Mutual Mean-Teaching
Unsupervised image segmentation aims at assigning the pixels with similar feature into a same cluster without annotation, which is an important task in computer vision.
LAP-Net: Adaptive Features Sampling via Learning Action Progression for Online Action Detection
At each time step, this sampling strategy first estimates current action progression and then decide what temporal ranges should be used to aggregate the optimal supplementary features.
Scope Head for Accurate Localization in Object Detection
Existing anchor-based and anchor-free object detectors in multi-stage or one-stage pipelines have achieved very promising detection performance.
Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis
We propose a novel ECGAN for the challenging semantic image synthesis task.
Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation
To tackle this issue, in this work we consider learning the scene generation in a local context, and correspondingly design a local class-specific generative network with semantic maps as a guidance, which separately constructs and learns sub-generators concentrating on the generation of different classes, and is able to provide more scene details.
AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks
State-of-the-art methods in image-to-image translation are capable of learning a mapping from a source domain to a target domain with unpaired image data.
Ranked #1 on
Facial Expression Translation
on CelebA
Progressive Fusion for Unsupervised Binocular Depth Estimation using Cycled Networks
Extensive experiments on the publicly available datasets KITTI, Cityscapes and ApolloScape demonstrate the effectiveness of the proposed model which is competitive with other unsupervised deep learning methods for depth prediction.
Structured Modeling of Joint Deep Feature and Prediction Refinement for Salient Object Detection
We formulate the CRF graphical model that involves message-passing of feature-feature, feature-prediction, and prediction-prediction, from the coarse scale to the finer scale, to update the features and the corresponding predictions.
Geometry-Aware Video Object Detection for Static Cameras
In this paper we propose a geometry-aware model for video object detection.
Dynamic Graph Message Passing Networks
We propose a dynamic graph message passing network, that significantly reduces the computational complexity compared to related works modelling a fully-connected graph.
Structured Coupled Generative Adversarial Networks for Unsupervised Monocular Depth Estimation
Inspired by the success of adversarial learning, we propose a new end-to-end unsupervised deep learning framework for monocular depth estimation consisting of two Generative Adversarial Networks (GAN), deeply coupled with a structured Conditional Random Field (CRF) model.
Monocular Depth Estimation
Unsupervised Monocular Depth Estimation
Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation
In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C$^2$GAN) for the task of keypoint-guided image generation.
Expression Conditional GAN for Facial Expression-to-Expression Translation
In this paper, we focus on the facial expression translation task and propose a novel Expression Conditional GAN (ECGAN) which can learn the mapping from one image domain to another one based on an additional expression attribute.
Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation
In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (SelectionGAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map.
Bird View Synthesis
Cross-View Image-to-Image Translation
+1
Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation
To handle the limitation, in this paper we propose a novel Attention-Guided Generative Adversarial Network (AGGAN), which can detect the most discriminative semantic object and minimize changes of unwanted part for semantic manipulation problems without using extra data and models.
Ranked #1 on
Facial Expression Translation
on CelebA
Attribute-Guided Sketch Generation
To this end, we propose a novel Attribute-Guided Sketch Generative Adversarial Network (ASGAN) which is an end-to-end framework and contains two pairs of generators and discriminators, one of which is used to generate faces with attributes while the other one is employed for image-to-sketch translation.
Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
State-of-the-art methods for image-to-image translation with Generative Adversarial Networks (GANs) can learn a mapping from one domain to another domain using unpaired image data.
Generative Adversarial Network
Image-to-Image Translation
+1
Deep Micro-Dictionary Learning and Coding Network
In this paper, we propose a novel Deep Micro-Dictionary Learning and Coding Network (DDLCN).
GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
Therefore, this task requires a high-level understanding of the mapping between the input source gesture and the output target gesture.
Ranked #1 on
Gesture-to-Gesture Translation
on NTU Hand Digit
Unsupervised Adversarial Depth Estimation using Cycled Generative Networks
The proposed architecture consists of two generative sub-networks jointly trained with adversarial learning for reconstructing the disparity map and organized in a cycle such as to provide mutual constraints and supervision to each other.
Group Consistent Similarity Learning via Deep CRF for Person Re-Identification
Extensive experiments demonstrate the effectiveness of our model that combines DNN and CRF for learning robust multi-scale local similarities.
PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
Depth estimation and scene parsing are two particularly important tasks in visual scene understanding.
Ranked #15 on
Depth Estimation
on NYU-Depth V2
Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation
Recent works have shown the benefit of integrating Conditional Random Fields (CRFs) models into deep architectures for improving pixel-level prediction tasks.
Cross-Paced Representation Learning with Partial Curricula for Sketch-based Image Retrieval
In this paper we address the problem of learning robust cross-domain representations for sketch-based image retrieval (SBIR).
Monocular Depth Estimation using Multi-Scale Continuous CRFs as Sequential Deep Networks
Depth cues have been proved very useful in various computer vision and robotic tasks.
Every Smile is Unique: Landmark-Guided Diverse Smile Generation
Finally, these landmark sequences are translated into face videos.
Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction
Recent works have shown that exploiting multi-scale representations deeply learned via convolutional neural networks (CNN) is of tremendous importance for accurate contour detection.
Learning Cross-Modal Deep Representations for Robust Pedestrian Detection
Then, the learned feature representations are transferred to a second deep network, which receives as input an RGB image and outputs the detection results.
Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
This paper addresses the problem of depth estimation from a single still image.
Ranked #14 on
Depth Estimation
on NYU-Depth V2
Viraliency: Pooling Local Virality
In our overly-connected world, the automatic recognition of virality - the quality of an image or video to be rapidly and widely spread in social networks - is of crucial importance, and has recently awaken the interest of the computer vision community.
Neural Information Retrieval: A Literature Review
A recent "third wave" of Neural Network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing.
Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking
In this paper, we address the problem of searching action proposals in unconstrained video clips.
Learning Deep Representations of Appearance and Motion for Anomalous Event Detection
We present a novel unsupervised deep learning framework for anomalous event detection in complex video scenes.
