Search Results for author:
Found 94 papers, 51 papers with code
Guided Semantic Flow
Establishing dense semantic correspondences requires dealing with large geometric variations caused by the unconstrained setting of images.
Cross-View Completion Models are Zero-shot Correspondence Estimators
In this work, we explore new perspectives on cross-view completion learning by drawing an analogy to self-supervised correspondence learning.
A Noise is Worth Diffusion Guidance
Observing that noise obtained via diffusion inversion can reconstruct high-quality images without guidance, we focus on the initial noise of the denoising pipeline.
Appearance Matching Adapter for Exemplar-based Semantic Image Synthesis
Recent tuning-free approaches address this limitation by transferring local appearance from the exemplar image to the synthesized image through implicit cross-image matching in the augmented self-attention mechanism of pre-trained diffusion models.
Multi-Granularity Video Object Segmentation
Thus, developing a new video segmentation dataset aimed at tracking multi-granularity segmentation target in the video scene is necessary.
Referring Video Object Segmentation via Language-aligned Track Selection
In this work, we present Selection by Object Language Alignment (SOLA), a novel framework that reformulates RVOS into two sub-problems, track generation and track selection.
ControlFace: Harnessing Facial Parametric Control for Face Rigging
Manipulation of facial images to meet specific controls such as pose, expression, and lighting, also known as face rigging, is a complex task in computer vision.
PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting
We then introduce lightweight, learnable modules to refine depth and pose estimates from the coarse alignments, improving the quality of 3D reconstruction and novel view synthesis.
Towards Open-Vocabulary Semantic Segmentation Without Semantic Labels
Large-scale vision-language models like CLIP have demonstrated impressive open-vocabulary capabilities for image-level tasks, excelling in recognizing what objects are present.
Local All-Pair Correspondence for Point Tracking
We introduce LocoTrack, a highly accurate and efficient model designed for the task of tracking any point (TAP) across video sequences.
Ranked #1 on
Point Tracking
on TAP-Vid-DAVIS-First
Geometry-Aware Score Distillation via 3D Consistent Noising and Gradient Consistency Modeling
We demonstrate that our method significantly improves performance, successfully addressing the geometric inconsistency problems in text-to-3D generation task with minimal computation cost and being compatible with existing score distillation-based models.
Pose-dIVE: Pose-Diversified Augmentation with Diffusion Model for Person Re-Identification
Person re-identification (Re-ID) often faces challenges due to variations in human poses and camera viewpoints, which significantly affect the appearance of individuals across images.
GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping
In these methods, an input view is geometrically warped to novel views with estimated depth maps, then the warped image is inpainted by T2I models.
GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splatting
A key insight is to encode the 3D Gaussian attributes into a shared implicit feature representation, where it is merged with audio features to manipulate each Gaussian attribute.
Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Prior
Given the personalized 3D generative model, we present a novel audio-guided attention U-Net architecture that predicts the dynamic face variations in the NeRF space driven by audio.
MoDiTalker: Motion-Disentangled Diffusion Model for High-Fidelity Talking Head Generation
AToM excels in capturing subtle lip movements by leveraging an audio attention mechanism.
Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance
These techniques are often not applicable in unconditional generation or in various downstream tasks such as image restoration.
Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence
In this work, we first show that feature aggregation and cost aggregation exhibit distinct characteristics and reveal the potential for substantial benefits stemming from the judicious use of both aggregation processes.
Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting
3D Gaussian splatting (3DGS) has recently demonstrated impressive capabilities in real-time novel view synthesis and 3D reconstruction.
HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning
Such fragmentation introduces a new type of data heterogeneity in FL, namely \textit{system-induced data heterogeneity}, as each device generates distinct data depending on its hardware and software configurations.
LatentSwap: An Efficient Latent Code Mapping Framework for Face Swapping
We show that our framework is applicable to other generators such as StyleNeRF, paving a way to 3D-aware face swapping and is also compatible with other downstream StyleGAN2 generator tasks.
Hybrid Video Diffusion Models with 2D Triplane and 3D Wavelet Representation
Generating high-quality videos that synthesize desired realistic content is a challenging task due to their intricate high-dimensionality and complexity of videos.
DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization
The objective of text-to-image (T2I) personalization is to customize a diffusion model to a user-provided reference concept, generating diverse images of the concept aligned with the target prompts.
Retrieval-Augmented Score Distillation for Text-to-3D Generation
We leverage the retrieved asset to incorporate its geometric prior in the variational objective and adapt the diffusion model's 2D prior toward view consistency, achieving drastic improvements in both geometry and fidelity of generated scenes.
Unifying Correspondence Pose and NeRF for Generalized Pose-Free Novel View Synthesis
This work delves into the task of pose-free novel view synthesis from stereo pairs a challenging and pioneering task in 3D vision.
FlowTrack: Revisiting Optical Flow for Long-Range Dense Tracking
In the domain of video tracking existing methods often grapple with a trade-off between spatial density and temporal range.
Context Enhanced Transformer for Single Image Object Detection
With the increasing importance of video data in real-world applications, there is a rising need for efficient object detection methods that utilize temporal information.
Universal Noise Annotation: Unveiling the Impact of Noisy annotation on Object Detection
For object detection task with noisy labels, it is important to consider not only categorization noise, as in image classification, but also localization noise, missing annotations, and bogus bounding boxes.
Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs
This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision.
Self-Evolving Neural Radiance Fields
We formulate few-shot NeRF into a teacher-student framework to guide the network to learn a more robust representation of the scene by training the student with additional pseudo labels generated from the teacher.
Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Images
This paper builds on the hypothesis that there is an inherent data-hungry matter in learning semantic correspondences and uncovers the models can be more trained by employing densified training pairs.
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters
Our focus in this study is on leveraging the knowledge of large pretrained models to improve handling of OOD scenarios and tackle domain generalization problems.
Ranked #1 on
Domain Generalization
on Office-Home
User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques
From the users' standpoint, prompt engineering is a labor-intensive process, and users prefer to provide a target word for editing instead of a full sentence.
Diffusion Model for Dense Matching
The objective for establishing dense correspondence between paired images consists of two terms: a data term and a prior term.
DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation
Employing monocular depth estimation (MDE) networks, pretrained on large-scale RGB-D datasets, with powerful generalization capability would be a key to solving this problem: however, using MDE in conjunction with NeRF comes with a new set of challenges due to various ambiguity problems exhibited by monocular depths.
DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation
In the paradigm of AI-generated content (AIGC), there has been increasing attention to transferring knowledge from pre-trained text-to-image (T2I) models to text-to-video (T2V) generation.
Panoramic Image-to-Image Translation
This task is difficult due to the geometric distortion of panoramic images and the lack of a panoramic image dataset with diverse conditions, like weather or time.
PartMix: Regularization Strategy to Learn Part Discovery for Visible-Infrared Person Re-identification
Modern data augmentation using a mixture-based technique can regularize the models from overfitting to the training data in various computer vision applications, but a proper data augmentation technique tailored for the part-based Visible-Infrared person Re-IDentification (VI-ReID) models remains unexplored.
Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation
In this work, we explore existing frameworks for score-distilling text-to-3D generation and identify the main causes of the view inconsistency problem -- the embedded bias of 2D diffusion models.
Few-shot Neural Radiance Fields Under Unconstrained Illumination
In this paper, we introduce a new challenge for synthesizing novel view images in practical environments with limited input multi-view images and varying lighting conditions.
CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation
Open-vocabulary semantic segmentation presents the challenge of labeling each pixel within an image based on a wide range of text descriptions.
Ranked #1 on
Open Vocabulary Semantic Segmentation
on ADE20K-150
Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
Text-to-3D generation has shown rapid progress in recent days with the advent of score distillation, a methodology of using pretrained text-to-2D diffusion models to optimize neural radiance field (NeRF) in the zero-shot setting.
GeCoNeRF: Few-shot Neural Radiance Fields via Geometric Consistency
We present a novel framework to regularize Neural Radiance Field (NeRF) in a few-shot setting with a geometry-aware consistency regularization.
You Truly Understand What I Need: Intellectual and Friendly Dialogue Agents grounding Knowledge and Persona
The agent selects the proper knowledge and persona to use for generating the answers with our candidate scoring implemented with a poly-encoder.
DiffFace: Diffusion-based Face Swapping with Facial Guidance
In this paper, we propose a diffusion-based face swapping framework for the first time, called DiffFace, composed of training ID conditional DDPM, sampling with facial guidance, and a target-preserving blending.
MaskingDepth: Masked Consistency Regularization for Semi-supervised Monocular Depth Estimation
We propose MaskingDepth, a novel semi-supervised learning framework for monocular depth estimation to mitigate the reliance on large ground-truth depth quantities.
DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models
Generative models have recently undergone significant advancement due to the diffusion models.
SplitNet: Learnable Clean-Noisy Label Splitting for Learning with Noisy Labels
To address this, some methods were proposed to automatically split clean and noisy labels, and learn a semi-supervised learner in a Learning with Noisy Labels (LNL) framework.
Controllable Style Transfer via Test-time Training of Implicit Neural Representation
We propose a controllable style transfer framework based on Implicit Neural Representation that pixel-wisely controls the stylized output via test-time training.
3D GAN Inversion with Pose Optimization
With the recent advances in NeRF-based 3D aware GANs quality, projecting an image into the latent space of these 3D-aware GANs has a natural advantage over 2D GAN inversion: not only does it allow multi-view consistent editing of the projected image, but it also enables 3D reconstruction and novel view synthesis when given only a single image.
Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence
Existing pipelines of semantic correspondence commonly include extracting high-level semantic features for the invariance against intra-class variations and background clutters.
Towards Flexible Inductive Bias via Progressive Reparameterization Scheduling
In addition, through Fourier analysis of feature maps, the model's response patterns according to signal frequency changes, we observe which inductive bias is advantageous for each data scale.
MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation
Specifically, we formulate a diffusion-based matching-and-generation framework that interleaves cross-domain matching and diffusion steps in the latent space by iteratively feeding the intermediate warp into the noising process and denoising it to generate a translated image.
Integrative Feature and Cost Aggregation with Transformers for Dense Correspondence
The current state-of-the-art are Transformer-based approaches that focus on either feature descriptors or cost volume aggregation.
Ranked #1 on
Geometric Matching
on HPatches
LANIT: Language-Driven Image-to-Image Translation for Unlabeled Data
Existing techniques for image-to-image translation commonly have suffered from two critical problems: heavy reliance on per-sample domain annotation and/or inability of handling multiple attributes per image.
ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization
We present a novel semi-supervised learning framework that intelligently leverages the consistency regularization between the model's predictions from two strongly-augmented views of an image, weighted by a confidence of pseudo-label, dubbed ConMatch.
Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation
However, the tokenization of a correlation map for transformer processing can be detrimental, because the discontinuity at token boundaries reduces the local context available near the token edges and decreases inductive bias.
Ranked #2 on
Semantic correspondence
on PF-WILLOW
AE-NeRF: Auto-Encoding Neural Radiance Fields for 3D-Aware Object Manipulation
We propose a novel framework for 3D-aware object manipulation, called Auto-Encoding Neural Radiance Fields (AE-NeRF).
Joint Learning of Feature Extraction and Cost Aggregation for Semantic Correspondence
In this paper, we propose a novel framework for jointly learning feature extraction and cost aggregation for semantic correspondence.
InstaFormer: Instance-Aware Image-to-Image Translation with Transformer
By augmenting such tokens with an instance-level feature extracted from the content feature with respect to bounding box information, our framework is capable of learning an interaction between object instances and the global image, thus boosting the instance-awareness.
Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels
In this paper, we present a simple, but effective solution for semantic correspondence that learns the networks in a semi-supervised manner by supplementing few ground-truth correspondences via utilization of a large amount of confident correspondences as pseudo-labels, called SemiMatch.
Semi-Supervised Learning with Mutual Distillation for Monocular Depth Estimation
We propose a semi-supervised learning framework for monocular depth estimation.
CATs++: Boosting Cost Aggregation with Convolutions and Transformers
Cost aggregation is a highly important process in image matching tasks, which aims to disambiguate the noisy matching scores.
Ranked #3 on
Semantic correspondence
on PF-PASCAL
AggMatch: Aggregating Pseudo Labels for Semi-Supervised Learning
In this paper, we address this limitation with a novel SSL framework for aggregating pseudo labels, called AggMatch, which refines initial pseudo labels by using different confident instances.
Memory-guided Image De-raining Using Time-Lapse Data
To address this problem, we propose a novel network architecture based on a memory network that explicitly helps to capture long-term rain streak information in the time-lapse data.
Cost Aggregation Is All You Need for Few-Shot Segmentation
We introduce a novel cost aggregation network, dubbed Volumetric Aggregation with Transformers (VAT), to tackle the few-shot segmentation task by using both convolutions and transformers to efficiently handle high dimensional correlation maps between query and support.
Ranked #3 on
Semantic correspondence
on PF-WILLOW
Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge
Humans usually have conversations by making use of prior knowledge about a topic and background information of the people whom they are talking to.
Deep Translation Prior: Test-time Training for Photorealistic Style Transfer
Recent techniques to solve photorealistic style transfer within deep convolutional neural networks (CNNs) generally require intensive training from large-scale datasets, thus having limited applicability and poor generalization ability to unseen images or styles.
HoughCL: Finding Better Positive Pairs in Dense Self-supervised Learning
Compared to previous works, our method shows better or comparable performance on dense prediction fine-tuning tasks.
MOI-Mixer: Improving MLP-Mixer with Multi Order Interactions in Sequential Recommendation
Successful sequential recommendation systems rely on accurately capturing the user's short-term and long-term interest.
Learning Canonical 3D Object Representation for Fine-Grained Recognition
By incorporating 3D shape and appearance jointly in a deep representation, our method learns the discriminative representation of the object and achieves competitive performance on fine-grained image recognition and vehicle re-identification.
Mining Better Samples for Contrastive Learning of Temporal Correspondence
We present a novel framework for contrastive learning of pixel-level representation using only unlabeled video.
Deep Matching Prior: Test-Time Optimization for Dense Correspondence
Conventional techniques to establish dense correspondences across visually or semantically similar images focused on designing a task-specific matching prior, which is difficult to model.
Ranked #1 on
Dense Pixel Correspondence Estimation
on HPatches
(using extra training data)
CATs: Cost Aggregation Transformers for Visual Correspondence
We propose a novel cost aggregation network, called Cost Aggregation Transformers (CATs), to find dense correspondences between semantically similar images with additional challenges posed by large intra-class appearance and geometric variations.
Ranked #5 on
Semantic correspondence
on PF-WILLOW
Modeling Object Dissimilarity for Deep Saliency Prediction
Saliency prediction has made great strides over the past two decades, with current techniques modeling low-level information, such as color, intensity and size contrasts, and high-level ones, such as attention and gaze direction for entire objects.
RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening
Enhancing the generalization capability of deep neural networks to unseen domains is crucial for safety-critical applications in the real world such as autonomous driving.
Ranked #5 on
Robust Object Detection
on DWD
On the confidence of stereo matching in a deep-learning era: a quantitative evaluation
Stereo matching is one of the most popular techniques to estimate dense depth maps by finding the disparity between matching pixels on two, synchronized and rectified images.
Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation
Existing techniques to adapt semantic segmentation networks across the source and target domains within deep convolutional neural networks (CNNs) deal with all the samples from the two domains in a global or category-aware manner.
Online Exemplar Fine-Tuning for Image-to-Image Translation
Existing techniques to solve exemplar-based image-to-image translation within deep convolutional neural networks (CNNs) generally require a training phase to optimize the network parameters on domain-specific and task-specific benchmarks, thus having limited applicability and generalization ability.
Adaptive confidence thresholding for monocular depth estimation
To cope with the prediction error of the confidence map itself, we also leverage the threshold network that learns the threshold dynamically conditioned on the pseudo depth maps.
Volumetric Transformer Networks
We design our VTN as an encoder-decoder network, with modules dedicated to letting the information flow across the feature channels, to account for the dependencies between the semantic parts.
Cylindrical Convolutional Networks for Joint Object Detection and Viewpoint Estimation
To overcome this limitation, we introduce a learnable module, cylindrical convolutional networks (CCNs), that exploit cylindrical representation of a convolutional kernel defined in the 3D space.
Joint Learning of Semantic Alignment and Object Landmark Detection
Based on the key insight that the two tasks can mutually provide supervisions to each other, our networks accomplish this through a joint loss function that alternatively imposes a consistency constraint between the two tasks, thereby boosting the performance and addressing the lack of training data in a principled manner.
Context-Aware Emotion Recognition Networks
We present deep networks for context-aware emotion recognition, called CAER-Net, that exploit not only human facial expression but also context information in a joint and boosting manner.
Ranked #1 on
Emotion Recognition in Context
on CAER-Dynamic
Semantic Attribute Matching Networks
SAM-Net accomplishes this through an iterative process of establishing reliable correspondences by reducing the attribute discrepancy between the images and synthesizing attribute transferred images using the learned correspondences.
Recurrent Transformer Networks for Semantic Correspondence
Our networks accomplish this through an iterative process of estimating spatial transformations between the input images and using these transformations to generate aligned convolutional activations.
PARN: Pyramidal Affine Regression Networks for Dense Semantic Correspondence
To the best of our knowledge, it is the first work that attempts to estimate dense affine transformation fields in a coarse-to-fine manner within deep networks.
DCTM: Discrete-Continuous Transformation Matching for Semantic Flow
In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor.
FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence
The sampling patterns of local structure and the self-similarity measure are jointly learned within the proposed network in an end-to-end and multi-scale manner.
DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation
In this paper, we propose a novel dense descriptor, called dense adaptive self-correlation (DASC), to estimate multi-modal and multi-spectral dense correspondences.
Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence
We present a novel descriptor, called deep self-convolutional activations (DeSCA), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions.
Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields
We present a method for jointly predicting a depth map and intrinsic images from single-image input.
DASC: Dense Adaptive Self-Correlation Descriptor for Multi-Modal and Multi-Spectral Correspondence
To further improve the matching quality and runtime efficiency, we propose a patch-wise receptive field pooling, in which a sampling pattern is optimized with a discriminative learning.
