Search Results for author:
Found 60 papers, 19 papers with code
GroPrompt: Efficient Grounded Prompting and Adaptation for Referring Video Object Segmentation
With the proposed TAP-CL, our GroPrompt framework can generate temporal-consistent yet text-aware position prompts describing locations and movements for the referred object from the video.
CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation
Recently video diffusion models have emerged as expressive generative tools for high-quality video content creation readily available to general users.
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model
Vision Language Models (VLMs) have demonstrated remarkable performance in 2D vision and language tasks.
Compositional Text-to-Image Generation with Dense Blob Representations
To leverage the compositionality of large language models (LLMs), we introduce a new in-context learning approach to generate blob representations from text prompts.
Communication-Efficient Collaborative Perception via Information Filling with Codebook
It inherently results in a fundamental trade-off between perception ability and communication cost.
HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data
In this paper, we propose HOIDiffusion for generating realistic and diverse 3D hand-object interaction data.
RegionGPT: Towards Region Understanding Vision Language Model
Vision language models (VLMs) have experienced rapid advancements through the integration of large language models (LLMs) with image-text pairs, yet they struggle with detailed regional visual understanding due to limited spatial awareness of the vision encoder, and the use of coarse-grained training data that lacks detailed, region-specific captions.
RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos
WildRGB-D comprises large-scale category-level RGB-D object videos, which are taken using an iPhone to go around the objects in 360 degrees.
AGG: Amortized Generative 3D Gaussians for Single Image to 3D
To overcome these challenges, we introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image, eliminating the need for per-instance optimization.
COLMAP-Free 3D Gaussian Splatting
While neural rendering has led to impressive advances in scene reconstruction and novel view synthesis, it relies heavily on accurately pre-computed camera poses.
A Unified Approach for Text- and Image-guided 4D Scene Generation
Large-scale diffusion generative models are greatly simplifying image, video and 3D asset creation from user-provided text prompts and images.
3D Reconstruction with Generalizable Neural Fields using Scene Priors
NFP not only demonstrates SOTA scene reconstruction performance and efficiency, but it also supports single-image novel-view synthesis, which is underexplored in neural fields.
Generalizable One-shot Neural Head Avatar
We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image.
Zero-shot Pose Transfer for Unrigged Stylized 3D Characters
We present a zero-shot approach that requires only the widely available deformed non-stylized avatars in training, and deforms stylized characters of significantly different shapes at inference.
TUVF: Learning Generalizable Texture UV Radiance Fields
This allows the texture to be disentangled from the underlying shape and transferable to other shapes that share the same UV space, i. e., from the same category.
Affordance Diffusion: Synthesizing Hand-Object Interactions
In contrast, in this work we focus on synthesizing complex interactions (ie, an articulated hand) with a given object.
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models
Our approach outperforms the previous state of the art by significant margins on both open-vocabulary panoptic and semantic segmentation tasks.
Ranked #2 on
Open-World Instance Segmentation
on UVO
(using extra training data)
Open Vocabulary Panoptic Segmentation
Open Vocabulary Semantic Segmentation
+4
Self-Supervised Super-Plane for Neural 3D Reconstruction
In this work, we introduce a self-supervised super-plane constraint by exploring the free geometry cues from the predicted surface, which can further regularize the reconstruction of plane regions without any other ground truth annotations.
Physics-based Indirect Illumination for Inverse Rendering
We present a physics-based inverse rendering method that learns the illumination, geometry, and materials of a scene from posed multi-view RGB images.
Autoregressive 3D Shape Generation via Canonical Mapping
With the capacity of modeling long-range dependencies in sequential data, transformers have shown remarkable performances in a variety of generative tasks such as image, audio, and text generation.
CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs
We represent the correspondence maps of different images as warped coordinate frames transformed from a canonical coordinate frame, i. e., the correspondence map, which describes the structure (e. g., the shape of a face), is controlled via a transformation.
GroupViT: Semantic Segmentation Emerges from Text Supervision
With only text supervision and without any pixel-level annotations, GroupViT learns to group together semantic regions and successfully transfers to the task of semantic segmentation in a zero-shot manner, i. e., without any further fine-tuning.
Coupled Segmentation and Edge Learning via Dynamic Graph Propagation
It is therefore interesting to study how these two tasks can be coupled to benefit each other.
Learning Continuous Environment Fields via Implicit Functions
We propose a novel scene representation that encodes reaching distance -- the distance between any position in the scene to a goal along a feasible trajectory.
Self-Supervised Object Detection via Generative Image Synthesis
We present SSOD, the first end-to-end analysis-by synthesis framework with controllable GANs for the task of self-supervised object detection.
Video Autoencoder: self-supervised disentanglement of static 3D structure and motion
Relying on temporal continuity in videos, our work assumes that the 3D scene structure in nearby video frames remains static.
Learning Contrastive Representation for Semantic Correspondence
Dense correspondence across semantically related images has been extensively studied, but still faces two challenges: 1) large variations in appearance, scale and pose exist even for objects from the same category, and 2) labeling pixel-level dense correspondences is labor intensive and infeasible to scale.
Learning 3D Dense Correspondence via Canonical Point Autoencoder
We propose a canonical point autoencoder (CPAE) that predicts dense correspondences between 3D shapes of the same category.
Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time
Estimating 3D hand and object pose from a single image is an extremely challenging problem: hands and objects are often self-occluded during interactions, and the 3D annotations are scarce as even humans cannot directly label the ground-truths from a single image perfectly.
Ranked #7 on
hand-object pose
on HO-3D v2
Contrastive Syn-to-Real Generalization
Training on synthetic data can be beneficial for label or data-scarce scenarios.
Learning to Track Instances without Video Annotations
Tracking segmentation masks of multiple instances has been intensively studied, but still faces two fundamental challenges: 1) the requirement of large-scale, frame-wise annotation, and 2) the complexity of two-stage approaches.
Video Matting via Consistency-Regularized Graph Neural Networks
In this paper, we propose to enhance the temporal coherence by Consistency-Regularized Graph Neural Networks (CRGNN) with the aid of a synthesized video matting dataset.
Learning Continuous Image Representation with Local Implicit Image Function
How to represent an image?
Ranked #2 on
Image Super-Resolution
on DIV2K val - 4x upscaling
(SSIM metric)
Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes
Synthesizing 3D human motion plays an important role in many graphics applications as well as understanding human activity.
Online Adaptation for Consistent Mesh Reconstruction in the Wild
This paper presents an algorithm to reconstruct temporally consistent 3D meshes of deformable object instances from videos in the wild.
Hierarchical Contrastive Motion Learning for Video Action Recognition
Thus, the motion features at higher levels are trained to gradually capture semantic dynamics and evolve more discriminative for action recognition.
Regularizing Meta-Learning via Gradient Dropout
With the growing attention on learning-to-learn new tasks using only a few examples, meta-learning has been widely used in numerous problems such as few-shot classification, reinforcement learning, and domain generalization.
Self-Supervised Viewpoint Learning From Image Collections
Training deep neural networks to estimate the viewpoint of objects requires large labeled training datasets.
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
To the best of our knowledge, we are the first to try and solve the single-view reconstruction problem without a category-specific template mesh or semantic keypoints.
Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning
In this paper, we propose an iterative algorithm to learn such pairwise relations, which consists of two branches, a unary segmentation network which learns the label probabilities for each pixel, and a pairwise affinity network which learns affinity matrix and refines the probability map generated from the unary network.
Joint-task Self-supervised Learning for Temporal Correspondence
Our learning process integrates two highly related tasks: tracking large image regions \emph{and} establishing fine-grained pixel-level associations between consecutive video frames.
Learning Propagation for Arbitrarily-structured Data
We experiment with semantic segmentation networks, where we use our propagation module to jointly train on different data -- images, superpixels and point clouds.
Few-Shot Viewpoint Estimation
Through extensive experimentation on the ObjectNet3D and Pascal3D+ benchmark datasets, we demonstrate that our framework, which we call MetaView, significantly outperforms fine-tuning the state-of-the-art models with few examples, and that the specific architectural innovations of our method are crucial to achieving good performance.
SCOPS: Self-Supervised Co-Part Segmentation
Parts provide a good intermediate representation of objects that is robust with respect to the camera, pose and appearance variations.
Ranked #4 on
Unsupervised Keypoint Estimation
on CUB
Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments
In order to predict valid affordances and learn possible 3D human poses in indoor scenes, we need to understand the semantic and geometric structure of a scene as well as its potential interactions with a human.
Context-Aware Synthesis and Placement of Object Instances
Learning to insert an object instance into an image in a semantically coherent manner is a challenging and interesting problem.
Rendering Portraitures from Monocular Camera and Beyond
Specifically, we first exploit Convolutional Neural Networks to estimate the relative depth and portrait segmentation maps from a single input image.
Learning Linear Transformations for Fast Arbitrary Style Transfer
Recent arbitrary style transfer methods transfer second order statistics from reference image onto content image via a multiplication between content image features and a transformation matrix, which is computed from features with a pre-determined algorithm.
Learning Dual Convolutional Neural Networks for Low-Level Vision
These problems usually involve the estimation of two components of the target signals: structures and details.
Switchable Temporal Propagation Network
Our approach is based on a temporal propagation network (TPN), which models the transition-related affinity between a pair of frames in a purely data-driven manner.
Learning Video-Story Composition via Recurrent Neural Network
In this paper, we propose a learning-based method to compose a video-story from a group of video clips that describe an activity or experience.
Learning Affinity via Spatial Propagation Network
Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be the output from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix.
Learning Affinity via Spatial Propagation Networks
Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be the output from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix.
Learning to Segment Instances in Videos with Spatial Propagation Network
In addition, we apply a filter on the refined score map that aims to recognize the best connected region using spatial and temporal consistencies in the video.
Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Despite rapid advances in face recognition, there remains a clear gap between the performance of still image-based face recognition and video-based face recognition, due to the vast difference in visual quality between the domains and the difficulty of curating diverse large-scale video datasets.
Face Parsing via Recurrent Propagation
Face parsing is an important problem in computer vision that finds numerous applications including recognition and editing.
Generative Face Completion
In this paper, we propose an effective face completion algorithm using a deep generative model.
Deep Cascaded Bi-Network for Face Hallucination
We present a novel framework for hallucinating faces of unconstrained poses and with very low resolution (face size as small as 5pxIOD).
Ranked #5 on
Image Super-Resolution
on VggFace2 - 8x upscaling
Multi-Objective Convolutional Learning for Face Labeling
This paper formulates face labeling as a conditional random field with unary and pairwise classifiers.
Structured Face Hallucination
Each face image is represented in terms of facial components, contours and smooth regions.
