Search Results for author:
Found 63 papers, 33 papers with code
Guided Semantic Flow
Establishing dense semantic correspondences requires dealing with large geometric variations caused by the unconstrained setting of images.
User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques
From the users' standpoint, prompt engineering is a labor-intensive process, and users prefer to provide a target word for editing instead of a full sentence.
DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation
Employing monocular depth estimation (MDE) networks, pretrained on large-scale RGB-D datasets, with powerful generalization capability would be a key to solving this problem: however, using MDE in conjunction with NeRF comes with a new set of challenges due to various ambiguity problems exhibited by monocular depths.
DiffMatch: Diffusion Model for Dense Matching
The objective for establishing dense correspondence between paired images consists of two terms: a data term and a prior term.
Large Language Models are Frame-level Directors for Zero-shot Text-to-Video Generation
In the paradigm of AI-generated content (AIGC), there has been increasing attention in extending pre-trained text-to-image (T2I) models to text-to-video (T2V) generation.
Panoramic Image-to-Image Translation
This task is difficult due to the geometric distortion of panoramic images and the lack of a panoramic image dataset with diverse conditions, like weather or time.
PartMix: Regularization Strategy to Learn Part Discovery for Visible-Infrared Person Re-identification
Modern data augmentation using a mixture-based technique can regularize the models from overfitting to the training data in various computer vision applications, but a proper data augmentation technique tailored for the part-based Visible-Infrared person Re-IDentification (VI-ReID) models remains unexplored.
Debiasing Scores and Prompts of 2D Diffusion for Robust Text-to-3D Generation
The view inconsistency problem in score-distilling text-to-3D generation, also known as the Janus problem, arises from the intrinsic bias of 2D diffusion models, which leads to the unrealistic generation of 3D objects.
CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation
However, the problem of transferring these capabilities learned from image-level supervision to the pixel-level task of segmentation and addressing arbitrary unseen categories at inference makes this task challenging.
ExtremeNeRF: Few-shot Neural Radiance Fields Under Unconstrained Illumination
In this paper, we propose a new challenge that synthesizes a novel view in a more practical environment, where the number of input multi-view images is limited and illumination variations are significant.
Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
Text-to-3D generation has shown rapid progress in recent days with the advent of score distillation, a methodology of using pretrained text-to-2D diffusion models to optimize neural radiance field (NeRF) in the zero-shot setting.
GeCoNeRF: Few-shot Neural Radiance Fields via Geometric Consistency
We present a novel framework to regularize Neural Radiance Field (NeRF) in a few-shot setting with a geometry-aware consistency regularization.
You Truly Understand What I Need: Intellectual and Friendly Dialogue Agents grounding Knowledge and Persona
The agent selects the proper knowledge and persona to use for generating the answers with our candidate scoring implemented with a poly-encoder.
DiffFace: Diffusion-based Face Swapping with Facial Guidance
In this paper, we propose a diffusion-based face swapping framework for the first time, called DiffFace, composed of training ID conditional DDPM, sampling with facial guidance, and a target-preserving blending.
MaskingDepth: Masked Consistency Regularization for Semi-supervised Monocular Depth Estimation
We propose MaskingDepth, a novel semi-supervised learning framework for monocular depth estimation to mitigate the reliance on large ground-truth depth quantities.
DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models
Generative models have recently undergone significant advancement due to the diffusion models.
SplitNet: Learnable Clean-Noisy Label Splitting for Learning with Noisy Labels
To address this, some methods were proposed to automatically split clean and noisy labels, and learn a semi-supervised learner in a Learning with Noisy Labels (LNL) framework.
Controllable Style Transfer via Test-time Training of Implicit Neural Representation
We propose a controllable style transfer framework based on Implicit Neural Representation that pixel-wisely controls the stylized output via test-time training.
3D GAN Inversion with Pose Optimization
With the recent advances in NeRF-based 3D aware GANs quality, projecting an image into the latent space of these 3D-aware GANs has a natural advantage over 2D GAN inversion: not only does it allow multi-view consistent editing of the projected image, but it also enables 3D reconstruction and novel view synthesis when given only a single image.
Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence
Existing pipelines of semantic correspondence commonly include extracting high-level semantic features for the invariance against intra-class variations and background clutters.
Towards Flexible Inductive Bias via Progressive Reparameterization Scheduling
In addition, through Fourier analysis of feature maps, the model's response patterns according to signal frequency changes, we observe which inductive bias is advantageous for each data scale.
Improving Sample Quality of Diffusion Models Using Self-Attention Guidance
Denoising diffusion models (DDMs) have attracted attention for their exceptional generation quality and diversity.
MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation
Specifically, we formulate a diffusion-based matching-and-generation framework that interleaves cross-domain matching and diffusion steps in the latent space by iteratively feeding the intermediate warp into the noising process and denoising it to generate a translated image.
Integrative Feature and Cost Aggregation with Transformers for Dense Correspondence
The current state-of-the-art are Transformer-based approaches that focus on either feature descriptors or cost volume aggregation.
Ranked #1 on
Geometric Matching
on HPatches
LANIT: Language-Driven Image-to-Image Translation for Unlabeled Data
Existing techniques for image-to-image translation commonly have suffered from two critical problems: heavy reliance on per-sample domain annotation and/or inability of handling multiple attributes per image.
ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization
We present a novel semi-supervised learning framework that intelligently leverages the consistency regularization between the model's predictions from two strongly-augmented views of an image, weighted by a confidence of pseudo-label, dubbed ConMatch.
Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation
However, the tokenization of a correlation map for transformer processing can be detrimental, because the discontinuity at token boundaries reduces the local context available near the token edges and decreases inductive bias.
Ranked #1 on
Semantic correspondence
on PF-WILLOW
AE-NeRF: Auto-Encoding Neural Radiance Fields for 3D-Aware Object Manipulation
We propose a novel framework for 3D-aware object manipulation, called Auto-Encoding Neural Radiance Fields (AE-NeRF).
Joint Learning of Feature Extraction and Cost Aggregation for Semantic Correspondence
In this paper, we propose a novel framework for jointly learning feature extraction and cost aggregation for semantic correspondence.
Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels
In this paper, we present a simple, but effective solution for semantic correspondence that learns the networks in a semi-supervised manner by supplementing few ground-truth correspondences via utilization of a large amount of confident correspondences as pseudo-labels, called SemiMatch.
InstaFormer: Instance-Aware Image-to-Image Translation with Transformer
By augmenting such tokens with an instance-level feature extracted from the content feature with respect to bounding box information, our framework is capable of learning an interaction between object instances and the global image, thus boosting the instance-awareness.
Semi-Supervised Learning with Mutual Distillation for Monocular Depth Estimation
We propose a semi-supervised learning framework for monocular depth estimation.
CATs++: Boosting Cost Aggregation with Convolutions and Transformers
Cost aggregation is a highly important process in image matching tasks, which aims to disambiguate the noisy matching scores.
Ranked #1 on
Semantic correspondence
on PF-PASCAL
AggMatch: Aggregating Pseudo Labels for Semi-Supervised Learning
In this paper, we address this limitation with a novel SSL framework for aggregating pseudo labels, called AggMatch, which refines initial pseudo labels by using different confident instances.
Memory-guided Image De-raining Using Time-Lapse Data
To address this problem, we propose a novel network architecture based on a memory network that explicitly helps to capture long-term rain streak information in the time-lapse data.
Cost Aggregation Is All You Need for Few-Shot Segmentation
We introduce a novel cost aggregation network, dubbed Volumetric Aggregation with Transformers (VAT), to tackle the few-shot segmentation task by using both convolutions and transformers to efficiently handle high dimensional correlation maps between query and support.
Ranked #2 on
Semantic correspondence
on PF-WILLOW
Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge
Humans usually have conversations by making use of prior knowledge about a topic and background information of the people whom they are talking to.
Deep Translation Prior: Test-time Training for Photorealistic Style Transfer
Recent techniques to solve photorealistic style transfer within deep convolutional neural networks (CNNs) generally require intensive training from large-scale datasets, thus having limited applicability and poor generalization ability to unseen images or styles.
HoughCL: Finding Better Positive Pairs in Dense Self-supervised Learning
Compared to previous works, our method shows better or comparable performance on dense prediction fine-tuning tasks.
MOI-Mixer: Improving MLP-Mixer with Multi Order Interactions in Sequential Recommendation
Successful sequential recommendation systems rely on accurately capturing the user's short-term and long-term interest.
Learning Canonical 3D Object Representation for Fine-Grained Recognition
By incorporating 3D shape and appearance jointly in a deep representation, our method learns the discriminative representation of the object and achieves competitive performance on fine-grained image recognition and vehicle re-identification.
Mining Better Samples for Contrastive Learning of Temporal Correspondence
We present a novel framework for contrastive learning of pixel-level representation using only unlabeled video.
Deep Matching Prior: Test-Time Optimization for Dense Correspondence
Conventional techniques to establish dense correspondences across visually or semantically similar images focused on designing a task-specific matching prior, which is difficult to model.
Ranked #1 on
Dense Pixel Correspondence Estimation
on HPatches
(using extra training data)
CATs: Cost Aggregation Transformers for Visual Correspondence
We propose a novel cost aggregation network, called Cost Aggregation Transformers (CATs), to find dense correspondences between semantically similar images with additional challenges posed by large intra-class appearance and geometric variations.
Ranked #2 on
Semantic correspondence
on PF-PASCAL
Modeling Object Dissimilarity for Deep Saliency Prediction
Saliency prediction has made great strides over the past two decades, with current techniques modeling low-level information, such as color, intensity and size contrasts, and high-level ones, such as attention and gaze direction for entire objects.
RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening
Enhancing the generalization capability of deep neural networks to unseen domains is crucial for safety-critical applications in the real world such as autonomous driving.
On the confidence of stereo matching in a deep-learning era: a quantitative evaluation
Stereo matching is one of the most popular techniques to estimate dense depth maps by finding the disparity between matching pixels on two, synchronized and rectified images.
Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation
Existing techniques to adapt semantic segmentation networks across the source and target domains within deep convolutional neural networks (CNNs) deal with all the samples from the two domains in a global or category-aware manner.
Online Exemplar Fine-Tuning for Image-to-Image Translation
Existing techniques to solve exemplar-based image-to-image translation within deep convolutional neural networks (CNNs) generally require a training phase to optimize the network parameters on domain-specific and task-specific benchmarks, thus having limited applicability and generalization ability.
Adaptive confidence thresholding for monocular depth estimation
To cope with the prediction error of the confidence map itself, we also leverage the threshold network that learns the threshold dynamically conditioned on the pseudo depth maps.
Volumetric Transformer Networks
We design our VTN as an encoder-decoder network, with modules dedicated to letting the information flow across the feature channels, to account for the dependencies between the semantic parts.
Cylindrical Convolutional Networks for Joint Object Detection and Viewpoint Estimation
To overcome this limitation, we introduce a learnable module, cylindrical convolutional networks (CCNs), that exploit cylindrical representation of a convolutional kernel defined in the 3D space.
Joint Learning of Semantic Alignment and Object Landmark Detection
Based on the key insight that the two tasks can mutually provide supervisions to each other, our networks accomplish this through a joint loss function that alternatively imposes a consistency constraint between the two tasks, thereby boosting the performance and addressing the lack of training data in a principled manner.
Context-Aware Emotion Recognition Networks
We present deep networks for context-aware emotion recognition, called CAER-Net, that exploit not only human facial expression but also context information in a joint and boosting manner.
Ranked #7 on
Emotion Recognition in Context
on EMOTIC
Semantic Attribute Matching Networks
SAM-Net accomplishes this through an iterative process of establishing reliable correspondences by reducing the attribute discrepancy between the images and synthesizing attribute transferred images using the learned correspondences.
Recurrent Transformer Networks for Semantic Correspondence
Our networks accomplish this through an iterative process of estimating spatial transformations between the input images and using these transformations to generate aligned convolutional activations.
PARN: Pyramidal Affine Regression Networks for Dense Semantic Correspondence
To the best of our knowledge, it is the first work that attempts to estimate dense affine transformation fields in a coarse-to-fine manner within deep networks.
DCTM: Discrete-Continuous Transformation Matching for Semantic Flow
In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor.
FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence
The sampling patterns of local structure and the self-similarity measure are jointly learned within the proposed network in an end-to-end and multi-scale manner.
DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation
In this paper, we propose a novel dense descriptor, called dense adaptive self-correlation (DASC), to estimate multi-modal and multi-spectral dense correspondences.
Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence
We present a novel descriptor, called deep self-convolutional activations (DeSCA), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions.
Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields
We present a method for jointly predicting a depth map and intrinsic images from single-image input.
DASC: Dense Adaptive Self-Correlation Descriptor for Multi-Modal and Multi-Spectral Correspondence
To further improve the matching quality and runtime efficiency, we propose a patch-wise receptive field pooling, in which a sampling pattern is optimized with a discriminative learning.
