Search Results for author:
Found 540 papers, 271 papers with code
Modeling the Effects of Windshield Refraction for Camera Calibration
Instead of using a generalized camera approach, we propose a novel approach to jointly optimize a traditional camera model, and a mathematical representation of the windshield’s surface.
Fixing Localization Errors to Improve Image Classification
This way, in an effort to fix localization errors, our loss provides an extra supervisory signal that helps the model to better discriminate between similar classes.
Investigating the Effectiveness of Cross-Attention to Unlock Zero-Shot Editing of Text-to-Video Diffusion Models
With recent advances in image and video diffusion models for content creation, a plethora of techniques have been proposed for customizing their generated content.
Self-Explainable Affordance Learning with Embodied Caption
In the field of visual affordance learning, previous methods mainly used abundant images or videos that delineate human behavior patterns to identify action possibility regions for object manipulation, with a variety of applications in robotic tasks.
Empowering Image Recovery_ A Multi-Attention Approach
Integration of attention mechanisms across feature and positional dimensions further enhances the recovery of fine details.
Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentation
A key challenge in panoptic UDA is reducing the domain gap between a labeled source and an unlabeled target domain while harmonizing the subtasks of semantic and instance segmentation to limit catastrophic interference.
Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning
We propose KYN, a novel method for single-view scene reconstruction that reasons about semantic and spatial context to predict each point's density.
HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud
Extracting keypoint locations from input hand frames, known as 3D hand pose estimation, is a critical task in various human-computer interaction applications.
I-Design: Personalized LLM Interior Designer
Interior design allows us to be who we are and live how we want - each design is as unique as our distinct personality.
A Unified and Interpretable Emotion Representation and Expression Generation
However, emotions are often compound, e. g. happily surprised, and can be mapped to the action units (AUs) used for expressing emotions, and trivially to the canonical ones.
GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM
To alleviate this issue, with the aid of a monocular depth estimator, we introduce a novel DSPO layer for bundle adjustment which optimizes the pose and depth of keyframes along with the scale of the monocular depth.
UniDepth: Universal Monocular Metric Depth Estimation
However, the remarkable accuracy of recent MMDE methods is confined to their training domains.
Ranked #2 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
Towards Online Real-Time Memory-based Video Inpainting Transformers
Video inpainting tasks have seen significant improvements in recent years with the rise of deep neural networks and, in particular, vision transformers.
FocusCLIP: Multimodal Subject-Level Guidance for Zero-Shot Transfer in Human-Centric Tasks
We propose FocusCLIP, integrating subject-level guidance--a specialized mechanism for target-specific supervision--into the CLIP framework for improved zero-shot transfer on human-centric tasks.
Ranked #1 on
Emotion Recognition
on EMOTIC
Rethinking Few-shot 3D Point Cloud Semantic Segmentation
The former arises from non-uniform point sampling, allowing models to distinguish the density disparities between foreground and background for easier segmentation.
Few-shot 3D Point Cloud Semantic Segmentation
Segmentation
+1
Loopy-SLAM: Dense Neural SLAM with Loop Closures
Neural RGBD SLAM techniques have shown promise in dense Simultaneous Localization And Mapping (SLAM), yet face challenges such as error accumulation during camera tracking resulting in distorted maps.
Cross-Domain Few-Shot Object Detection via Enhanced Open-Set Object Detector
This paper studies the challenging cross-domain few-shot object detection (CD-FSOD), aiming to develop an accurate object detector for novel domains with minimal labeled examples.
Key-Graph Transformer for Image Restoration
While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution.
Image Fusion via Vision-Language Model
Therefore, we introduce a novel fusion paradigm named image Fusion via vIsion-Language Model (FILM), for the first time, utilizing explicit textual information in different source images to guide image fusion.
Vanishing-Point-Guided Video Semantic Segmentation of Driving Scenes
The estimation of implicit cross-frame correspondences and the high computational cost have long been major challenges in video semantic segmentation (VSS) for driving scenes.
MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty
Achieving level-5 driving automation in autonomous vehicles necessitates a robust semantic visual perception system capable of parsing data from different sensors across diverse conditions.
Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation
Finally, we propose a novel self-guided pre-training method for graph representation learning.
Generative Adversarial Network
Graph Representation Learning
+1
InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes
We introduce InseRF, a novel method for generative object insertion in the NeRF reconstructions of 3D scenes.
Learning to Prompt with Text Only Supervision for Vision-Language Models
While effective, most of these works require labeled data which is not practical, and often struggle to generalize towards new datasets due to over-fitting on the source data.
Residual Learning for Image Point Descriptors
Such an approach of optimization allows us to discard learning knowledge already present in non-differentiable functions such as the hand-crafted descriptors and only learn the residual knowledge in the main network branch.
Ternary-type Opacity and Hybrid Odometry for RGB-only NeRF-SLAM
To foster this line of research, we also propose a simple yet novel visual odometry scheme that uses a hybrid combination of volumetric and warping-based image renderings.
Diffusion-Based Particle-DETR for BEV Perception
For the enhanced safety of AVs, modeling perception uncertainty in BEV is crucial.
G-MEMP: Gaze-Enhanced Multimodal Ego-Motion Prediction in Driving
In contrast, we focus on inferring the ego trajectory of a driver's vehicle using their gaze data.
DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control
Large, pretrained latent diffusion models (LDMs) have demonstrated an extraordinary ability to generate creative content, specialize to user data through few-shot fine-tuning, and condition their output on other modalities, such as semantic maps.
Zero-Shot Point Cloud Registration
The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods.
LALM: Long-Term Action Anticipation with Language Models
Understanding human activity is a crucial yet intricate task in egocentric vision, a field that focuses on capturing visual perspectives from the camera wearer's viewpoint.
Continuous Pose for Monocular Cameras in Neural Implicit Representation
In this paper, we showcase the effectiveness of optimizing monocular camera poses as a continuous function of time.
2D Feature Distillation for Weakly- and Semi-Supervised 3D Semantic Segmentation
As 3D perception problems grow in popularity and the need for large-scale labeled datasets for LiDAR semantic segmentation increase, new methods arise that aim to reduce the necessity for dense annotations by employing weakly-supervised training.
Single-Model and Any-Modality for Video Object Tracking
In practice, most existing RGB trackers learn a single set of parameters to use them across datasets and applications.
SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance
In SemiVL, we propose to integrate rich priors from VLM pre-training into semi-supervised semantic segmentation to learn better semantic decision boundaries.
Ranked #1 on
Semi-Supervised Semantic Segmentation
on PASCAL VOC 2012 732 labeled
(using extra training data)
Lego: Learning to Disentangle and Invert Concepts Beyond Object Appearance in Text-to-Image Diffusion Models
To enable customized content creation based on a few example images of a concept, methods such as Textual Inversion and DreamBooth invert the desired concept and enable synthesizing it in new scenes.
3D Compression Using Neural Fields
Neural Fields (NFs) have gained momentum as a tool for compressing various data modalities - e. g. images and videos.
Deep Equilibrium Diffusion Restoration with Parallel Sampling
Due to the inherent property of diffusion models, most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs.
Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions
In this work, we propose to predict 3D eye gaze from weak supervision of eye semantic segmentation masks and direct supervision of a few 3D gaze vectors.
MoVideo: Motion-Aware Video Generation with Diffusion Models
While recent years have witnessed great progress on using diffusion models for video generation, most of them are simple extensions of image generation frameworks, which fail to explicitly consider one of the key differences between videos and images, i. e., motion.
Contrastive Learning for Multi-Object Tracking with Transformers
The DEtection TRansformer (DETR) opened new possibilities for object detection by modeling it as a translation task: converting image features into object-level representations.
Ranked #1 on
Multiple Object Tracking
on BDD100K test
Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images
The key ideas presented in this paper are (i) Recovering accurate camera parameters via a robust pipeline from unposed day-to-day images is equally crucial in neural novel view synthesis problem; (ii) It is rather more practical to model object's content at different resolutions since dramatic camera motion is highly likely in day-to-day unposed images.
Long-Term Invariant Local Features via Implicit Cross-Domain Correspondences
Our proposed method enhances cross-domain localization performance, significantly reducing the performance gap.
Towards High-quality HDR Deghosting with Conditional Diffusion Models
To address this challenge, we formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition, consisting of the feature condition generator and the noise predictor.
SILC: Improving Vision Language Pretraining with Self-Distillation
However, the contrastive objective used by these models only focuses on image-text alignment and does not incentivise image feature learning for dense prediction tasks.
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
The real-world deployment of an autonomous driving system requires its components to run on-board and in real-time, including the motion prediction module that predicts the future trajectories of surrounding traffic participants.
Probabilistic Sampling of Balanced K-Means using Adiabatic Quantum Computing
Adiabatic quantum computing (AQC) is a promising quantum computing approach for discrete and often NP-hard optimization problems.
Discwise Active Learning for LiDAR Semantic Segmentation
Finally we propose a semi-supervised learning approach to utilize all frames within our dataset and improve performance.
Breathing New Life into 3D Assets with Generative Repainting
Diffusion-based text-to-image models ignited immense attention from the vision community, artists, and content creators.
Deformable Neural Radiance Fields using RGB and Event Cameras
In this work, we develop a novel method to model the deformable neural radiance fields using RGB and event cameras.
Temporal-aware Hierarchical Mask Classification for Video Semantic Segmentation
Modern approaches have proved the huge potential of addressing semantic segmentation as a mask classification task which is widely used in instance-level segmentation.
Three Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding
A common formulation to tackle 3D visual grounding is grounding-by-detection, where localization is done via bounding boxes.
Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving
VTD is a promising new direction for exploring the unification of perception tasks in autonomous driving.
Neural Gradient Regularizer
NGR is applicable to various image types and different image processing tasks, functioning in a zero-shot learning fashion, making it a versatile and plug-and-play regularizer.
Introducing Language Guidance in Prompt-based Continual Learning
While the model faces a disjoint set of classes in each task in this setting, we argue that these classes can be encoded to the same embedding space of a pre-trained language encoder.
DiffI2I: Efficient Diffusion Model for Image-to-Image Translation
Compared to traditional DMs, the compact IPR enables DiffI2I to obtain more accurate outcomes and employ a lighter denoising network and fewer iterations.
DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation
VLN-CE is a recently released embodied task, where AI agents need to navigate a freely traversable environment to reach a distant target location, given language instructions.
When Super-Resolution Meets Camouflaged Object Detection: A Comparison Study
Super Resolution (SR) and Camouflaged Object Detection (COD) are two hot topics in computer vision with various joint applications.
How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges
Google's Bard has emerged as a formidable competitor to OpenAI's ChatGPT in the field of conversational AI.
Prior Based Online Lane Graph Extraction from Single Onboard Camera Image
Thus, online estimation of the lane graph is crucial for widespread and reliable autonomous navigation.
Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis
To tackle 2), we design an effective module to selectively highlight class-dependent feature maps according to the original semantic layout to preserve the semantic information.
Improving Online Lane Graph Extraction by Object-Lane Clustering
In this work, we propose an architecture and loss formulation to improve the accuracy of local lane graph estimates by using 3D object detection outputs.
AutoDecoding Latent 3D Diffusion Models
We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core.
Prompting Diffusion Representations for Cross-Domain Semantic Segmentation
Intrigued by this result, we set out to explore how well diffusion-pretrained representations generalize to new domains, a crucial ability for any representation.
Unbalanced Optimal Transport: A Unified Framework for Object Detection
The approach is well suited for GPU implementation, which proves to be an advantage for large-scale models.
Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023
We present Palm, a solution to the Long-Term Action Anticipation (LTA) task utilizing vision-language and large language models.
UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM
We present an uncertainty learning framework for dense neural simultaneous localization and mapping (SLAM).
SF-FSDA: Source-Free Few-Shot Domain Adaptive Object Detection with Efficient Labeled Data Factory
Domain adaptive object detection aims to leverage the knowledge learned from a labeled source domain to improve the performance on an unlabeled target domain.
Condition-Invariant Semantic Segmentation
Motivated by these findings, we propose to leverage stylization in performing feature-level adaptation by aligning the internal network features extracted by the encoder of the network from the original and the stylized view of each input image with a novel feature invariance loss.
Equivariant Multi-Modality Image Fusion
These components enable the net training to follow the principles of the natural sensing-imaging process while satisfying the equivariant imaging prior.
Denoising Diffusion Models for Plug-and-Play Image Restoration
Although diffusion models have shown impressive performance for high-quality image synthesis, their potential to serve as a generative denoiser prior to the plug-and-play IR methods remains to be further explored.
StyleGenes: Discrete and Efficient Latent Distributions for GANs
Thus, by independently sampling a variant for each gene and combining them into the final latent vector, our approach can represent a vast number of unique latent samples from a compact set of learnable parameters.
Event-Free Moving Object Segmentation from Moving Ego Vehicle
Subsequently, we devise EmoFormer, a novel network able to exploit the event data.
Neural Implicit Dense Semantic SLAM
Visual Simultaneous Localization and Mapping (vSLAM) is a widely used technique in robotics and computer vision that enables a robot to create a map of an unfamiliar environment using a camera sensor while simultaneously tracking its position over time.
EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation
EDAPS significantly improves the state-of-the-art performance for panoptic segmentation UDA by a large margin of 20% on SYNTHIA-to-Cityscapes and even 72% on the more challenging SYNTHIA-to-Mapillary Vistas.
Ranked #1 on
Domain Adaptation
on Panoptic SYNTHIA-to-Mapillary
Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation
As previous UDA&DG semantic segmentation methods are mostly based on outdated networks, we benchmark more recent architectures, reveal the potential of Transformers, and design the DAFormer network tailored for UDA&DG.
Indiscernible Object Counting in Underwater Scenes
We further advance the frontier of this field by systematically studying a new challenge named indiscernible object counting (IOC), the goal of which is to count objects that are blended with respect to their surroundings.
Advances in Deep Concealed Scene Understanding
Concealed scene understanding (CSU) is a hot computer vision topic aiming to perceive objects exhibiting camouflage.
Quantum Annealing for Single Image Super-Resolution
As a result, in this work, we take the privilege to perform an early exploration of applying a quantum computing algorithm to this important image enhancement problem, i. e., SISR.
SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders
In this work, we propose a novel semi-supervised approach to realize few-shot HDR imaging via two stages of training, called SSHDR.
SAM Struggles in Concealed Scenes -- Empirical Study on "Segment Anything"
Segmenting anything is a ground-breaking step toward artificial general intelligence, and the Segment Anything Model (SAM) greatly fosters the foundation models for computer vision.
CamDiff: Camouflage Image Augmentation via Diffusion Model
Specifically, we leverage the latent diffusion model to synthesize salient objects in camouflaged scenes, while using the zero-shot image classification ability of the Contrastive Language-Image Pre-training (CLIP) model to prevent synthesis failures and ensure the synthesized object aligns with the input prompt.
Point-SLAM: Dense Neural Point Cloud-based SLAM
We propose a dense neural simultaneous localization and mapping (SLAM) approach for monocular RGBD input which anchors the features of a neural scene representation in a point cloud that is iteratively generated in an input-dependent data-driven manner.
Online Lane Graph Extraction from Onboard Video
One of the most common and useful representation of such an understanding is done in the form of BEV lane graphs.
Single Image Depth Prediction Made Better: A Multivariate Gaussian Take
Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution.
Enhanced Stable View Synthesis
Extensive evaluation of our approach on the popular benchmark dataset, such as Tanks and Temples, shows substantial improvement in view synthesis results compared to the prior art.
Unsupervised Deep Probabilistic Approach for Partial Point Cloud Registration
Deep point cloud registration methods face challenges to partial overlaps and rely on labeled data.
NeRF-GAN Distillation for Efficient 3D-Aware Generation with Convolutions
Pose-conditioned convolutional generative models struggle with high-quality 3D-consistent image generation from single-view datasets, due to their lack of sufficient 3D priors.
Lidar Line Selection with Spatially-Aware Shapley Value for Cost-Efficient Depth Completion
Lidar is a vital sensor for estimating the depth of a scene.
DiffIR: Efficient Diffusion Model for Image Restoration
Diffusion model (DM) has achieved SOTA performance by modeling the image synthesis process into a sequential application of a denoising network.
Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution
Then, the extracted features are mapped to the spherical space to complete the separation of private features and the alignment of shared features.
Graph Transformer GANs for Graph-Constrained House Generation
We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM).
Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation
We investigate normal-to-adverse condition model adaptation for semantic segmentation, whereby image-level correspondences are available in the target domain.
Ranked #1 on
Source-Free Domain Adaptation
on Cityscapes to ACDC
TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles.
A Multiplicative Value Function for Safe and Efficient Reinforcement Learning
To this end, we propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic.
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
The aim of this paper is to propose a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration.
Ranked #1 on
Image Defocus Deblurring
on DPD (Dual-view)
VA-DepthNet: A Variational Approach to Single Image Depth Prediction
While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene.
Ranked #18 on
Monocular Depth Estimation
on NYU-Depth V2
No One Left Behind: Real-World Federated Class-Incremental Learning
These issues render global model to undergo catastrophic forgetting on old categories, when local clients receive new categories consecutively under limited memory of storing old categories.
Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation
This task requires an understanding of the spatio-temporal context formed by past actions on objects, coined action context.
Event-Based Frame Interpolation with Ad-hoc Deblurring
The performance of video frame interpolation is inherently correlated with the ability to handle motion in the input scene.
Continuous Pseudo-Label Rectified Domain Adaptive Semantic Segmentation With Implicit Neural Representations
Unsupervised domain adaptation (UDA) for semantic segmentation aims at improving the model performance on the unlabeled target domain by leveraging a labeled source domain.
Self-Supervised Burst Super-Resolution
Extensive experiments on real and synthetic data show that, despite only using noisy bursts during training, models trained with our self-supervised strategy match, and sometimes surpass, the quality of fully-supervised baselines trained with synthetic data or weakly-paired ground-truth.
One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers
Thus, we aim to relieve this need on a large number of real data, and explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization (OSDG) problem, where only one real-world data sample is available.
CamoFormer: Masked Separable Attention for Camouflaged Object Detection
How to identify and segment camouflaged objects from the background is challenging.
Source-free Depth for Object Pop-out
In this work, we adapt such depth inference models for object segmentation using the objects' "pop-out" prior in 3D.
CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution
We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features.
I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification
Our proposed model, I2MVFormer, learns multi-view semantic embeddings for zero-shot image classification with these class views.
Surface Normal Clustering for Implicit Representation of Manhattan Scenes
In this work, we aim to leverage the geometric prior of Manhattan scenes to improve the implicit neural radiance field representations.
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation
MIC significantly improves the state-of-the-art performance across the different recognition tasks for synthetic-to-real, day-to-nighttime, and clear-to-adverse-weather UDA.
Knowledge Distillation based Degradation Estimation for Blind Super-Resolution
It consists of a knowledge distillation based implicit degradation estimator network (KD-IDE) and an efficient SR network.
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
We then introduce a dual-branch Transformer-CNN feature extractor with Lite Transformer (LT) blocks leveraging long-range attention to handle low-frequency global features and Invertible Neural Networks (INN) blocks focusing on extracting high-frequency local information.
DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models
Scene extrapolation -- the idea of generating novel views by flying into a given image -- is a promising, yet challenging task.
Ranked #1 on
Perpetual View Generation
on LHQ
Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images
On the one hand, the proposed method learns to segment these planar hulls from the labeled data.
Advancing Learned Video Compression with In-loop Frame Prediction
In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module, which is able to effectively predict the target frame from the previously compressed frames, without consuming any bit-rate.
MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning
While neural networks-based photo processing solutions can provide a better image quality compared to the traditional ISP systems, their application to mobile devices is still very limited due to their very high computational complexity.
PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks
The increased importance of mobile photography created a need for fast and performant RAW image processing pipelines capable of producing good visual results in spite of the mobile camera sensor limitations.
Towards Versatile Embodied Navigation
With the emergence of varied visual navigation tasks (e. g, image-/object-/audio-goal and vision-language navigation) that specify the target in different ways, the community has made appealing advances in training specialized agents capable of handling individual navigation tasks well.
TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM
We start from an off-the-shelf 3D object detector, and apply a tracking mechanism where objects are matched by an affinity score computed on local object feature embeddings and motion descriptors.
Masked Vision-Language Transformer in Fashion
We present a masked vision-language transformer (MVLT) for fashion-specific multi-modal representation.
Learning Attention Propagation for Compositional Zero-Shot Learning
CAPE learns to identify this structure and propagates knowledge between them to learn class embedding for all seen and unseen compositions.
Multi-View Photometric Stereo Revisited
The proposed approach in this paper exploits the benefit of uncertainty modeling in a deep neural network for a reliable fusion of photometric stereo (PS) and multi-view stereo (MVS) network predictions.
Composite Learning for Robust and Effective Dense Predictions
In this paper, we find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
Ranked #102 on
Semantic Segmentation
on NYU Depth v2
SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction
NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks, i. e., reconstructing real-world scenes and registering camera parameters simultaneously.
Robustifying the Multi-Scale Representation of Neural Radiance Fields
Although recently proposed Mip-NeRF could handle multi-scale imaging problems with NeRF, it cannot handle camera pose estimation error.
Basic Binary Convolution Unit for Binarized Image Restoration Network
In this study, we reconsider components in binary convolution, such as residual connection, BatchNorm, activation function, and structure, for IR tasks.
Physical Adversarial Attack meets Computer Vision: A Decade Survey
Building upon this foundation, we uncover the pervasive role of artifacts carrying adversarial perturbations in the physical world.
TT-NF: Tensor Train Neural Fields
Learning neural fields has been an active topic in deep learning research, focusing, among other issues, on finding more compact and easy-to-fit representations.
Exploiting Instance-based Mixed Sampling via Auxiliary Source Domain Supervision for Domain-adaptive Action Detection
We propose a novel domain adaptive action detection approach and a new adaptation protocol that leverages the recent advancements in image-level unsupervised domain adaptation (UDA) techniques and handle vagaries of instance-level video data.
I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification
In order to distill discriminative visual words from noisy documents, we introduce a new cross-modal attention module that learns fine-grained interactions between image patches and document words.
Spatio-Temporal Action Detection Under Large Motion
Current methods for spatiotemporal action tube detection often extend a bounding box proposal at a given keyframe into a 3D temporal cuboid and pool features from nearby frames.
Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising
To this end, we propose a new multi-scale refined optical flow-guided video denoising method, which is more robust to different noise levels.
Ranked #1 on
Video Denoising
on VideoLQ
ManiFlow: Implicitly Representing Manifolds with Normalizing Flows
In contrast to prior work, we approach this problem by generating samples from the original data distribution given full knowledge about the perturbed distribution and the noise model.
AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility
While being greatly benefiting to the tracking research, existing benchmarks do not pose the same difficulty as before with recent trackers achieving higher performance mainly due to (i) the introduction of more sophisticated transformers-based methods and (ii) the lack of diverse scenarios with adverse visibility such as, severe weather conditions, camouflage and imaging effects.
Reference-based Image Super-Resolution with Deformable Attention Transformer
Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images.
Towards Interpretable Video Super-Resolution via Alternating Optimization
These issues can be alleviated by a cascade of three separate sub-tasks, including video deblurring, frame interpolation, and super-resolution, which, however, would fail to capture the spatial and temporal correlations among video sequences.
Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation
The essence of video semantic segmentation (VSS) is how to leverage temporal information for prediction.
Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions
Due to the scarcity of dense pixel-level semantic annotations for images recorded in adverse visual conditions, there has been a keen interest in unsupervised domain adaptation (UDA) for the semantic segmentation of such images.
Ranked #1 on
Semantic Segmentation
on Dark Zurich
Organic Priors in Non-Rigid Structure from Motion
Besides that, the paper provides insights into the NRSfM factorization -- both in terms of shape and motion -- and is the first approach to show the benefit of single rotation averaging for NRSfM.
OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers
We present OSFormer, the first one-stage transformer framework for camouflaged instance segmentation (CIS).
L2E: Lasers to Events for 6-DoF Extrinsic Calibration of Lidars and Event Cameras
As neuromorphic technology is maturing, its application to robotics and autonomous vehicle systems has become an area of active research.
HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection
Besides standard cameras, autonomous vehicles typically include multiple additional sensors, such as lidars and radars, which help acquire richer information for perceiving the content of the driving scene.
Ranked #1 on
2D Object Detection
on Clear Weather
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation
Adapting to a continuously evolving environment is a safety-critical challenge inevitably faced by all autonomous driving systems.
Structured Sparsity Learning for Efficient Video Super-Resolution
In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks.
Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation
This paper presents MaskDistill: a novel framework for unsupervised semantic segmentation based on three key ideas.
Ranked #4 on
Unsupervised Semantic Segmentation
on PASCAL VOC 2012 val
(using extra training data)
Recurrent Video Restoration Transformer with Guided Deformable Attention
Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature.
Gradient Obfuscation Checklist Test Gives a False Sense of Security
It has since become a trend to use these five characteristics as a sufficient test, to determine whether or not gradient obfuscation is the main source of robustness.
GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector
In this paper, we present a novel end-to-end group collaborative learning network, termed GCoNet+, which can effectively and efficiently (250 fps) identify co-salient objects in natural scenes.
Ranked #1 on
Co-Salient Object Detection
on CoCA
Deep Gradient Learning for Efficient Camouflaged Object Detection
This paper introduces DGNet, a novel deep framework that exploits object gradient supervision for camouflaged object detection (COD).
Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement.
Ranked #1 on
Spectral Reconstruction
on Real HSI
Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration
On the other hand, we equip the sequence-to-sequence model with an unsupervised optical flow estimator to maximize its potential.
Ranked #2 on
Video Enhancement
on MFQE v2
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials
Within the proposed benchmark, we explore some commonly known essentials of standard continual learning.
Revisiting Random Channel Pruning for Neural Network Compression
The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning.
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation
Therefore, we propose HRDA, a multi-resolution training approach for UDA, that combines the strengths of small high-resolution crops to preserve fine segmentation details and large low-resolution crops to capture long-range context dependencies with a learned scale attention, while maintaining a manageable GPU memory footprint.
Ranked #3 on
Semantic Segmentation
on GTAV-to-Cityscapes Labels
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction
Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).
Ranked #1 on
Spectral Reconstruction
on ARAD-1K
Neural Vector Fields for Implicit Surface Representation and Inference
With neural networks, several other variations and training principles have been proposed with the goal to represent all classes of shapes.
Learning Local and Global Temporal Contexts for Video Semantic Segmentation
Specifically, we uniformly sample certain frames from the video and extract global contextual prototypes by k-means.
Learning Online Multi-Sensor Depth Fusion
Multi-sensor depth fusion is able to substantially improve the robustness and accuracy of 3D reconstruction methods, but existing techniques are not robust enough to handle sensors which operate with diverse value ranges as well as noise and outlier statistics.
Arbitrary-Scale Image Synthesis
Positional encodings have enabled recent works to train a single adversarial network that can generate images of different scales.
Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Models
Fundamental to the success of our method is the design of a novel multi-agent policy network that can steer a vehicle given the state of the surrounding agents and the map information.
P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior
We focus on the supervised setup, in which ground-truth depth is available only at training time.
Ranked #6 on
Depth Estimation
on NYU-Depth V2
FoV-Net: Field-of-View Extrapolation Using Self-Attention and Uncertainty
The ability to make educated predictions about their surroundings, and associate them with certain confidence, is important for intelligent systems, like autonomous vehicles and robots.
Direct Dense Pose Estimation
In addition, DDP is computationally more efficient than previous dense pose estimation methods, and it reduces jitters when applied to a video sequence, which is a problem plaguing the previous methods.
Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation
Since the rise of vision-language navigation (VLN), great progress has been made in instruction following -- building a follower to navigate environments under the guidance of instructions.
LiDAR Snowfall Simulation for Robust 3D Object Detection
Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds.
Ranked #1 on
3D Object Detection
on Heavy Snowfall
Rethinking Semantic Segmentation: A Prototype View
Prevalent semantic segmentation solutions, despite their different network designs (FCN based or attention based) and mask decoding strategies (parametric softmax based or pixel-query based), can be placed in one category, by considering the softmax weights or query vectors as learnable class prototypes.
Video Polyp Segmentation: A Deep Learning Perspective
We present the first comprehensive video polyp segmentation (VPS) study in the deep learning era.
Ranked #2 on
Video Polyp Segmentation
on SUN-SEG-Easy (Unseen)
Spatially Multi-conditional Image Generation
However, multi-conditional image generation is a very challenging problem due to the heterogeneity and the sparsity of the (in practice) available conditioning labels.
Continual Test-Time Domain Adaptation
However, real-world machine perception systems are running in non-stationary and continually changing environments where the target domain distribution can change over time.
Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis
While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved.
Ranked #1 on
Image Denoising
on urban100 sigma15
Transforming Model Prediction for Tracking
Optimization based tracking methods have been widely successful by integrating a target model prediction module, providing effective global reasoning by minimizing an objective function.
Ranked #20 on
Visual Object Tracking
on LaSOT
(Precision metric)
Robust Visual Tracking by Segmentation
We infer a bounding box from the segmentation mask, validate our tracker on challenging tracking datasets and achieve the new state of the art on LaSOT with a success AUC score of 69. 7%.
Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild
We propose a trainable Image Signal Processing (ISP) framework that produces DSLR quality images given RAW images captured by a smartphone.
Scribble-Supervised LiDAR Semantic Segmentation
Densely annotating LiDAR point clouds remains too expensive and time-consuming to keep up with the ever growing volume of data.
Ranked #1 on
3D Semantic Segmentation
on ScribbleKITTI
Zero Pixel Directional Boundary by Vector Transform
One of the key problems in boundary detection is the label representation, which typically leads to class imbalance and, as a consequence, to thick boundaries that require non-differential post-processing steps to be thinned.
Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound
Benefited from this new perspective, we first propose a new deep semi-supervised learning framework called Semi-supervised Learning by Empirical Distribution Alignment (SLEDA), in which existing technologies from the domain adaptation community can be readily used to address the semi-supervised learning problem through reducing the empirical distribution distance between labeled and unlabeled data.
Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction
Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i. e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement.
Ranked #2 on
Spectral Reconstruction
on Real HSI
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences
We propose Probabilistic Warp Consistency, a weakly-supervised learning objective for semantic matching.
Barlow constrained optimization for Visual Question Answering
In this paper, we propose a novel regularization for VQA models, Constrained Optimization using Barlow's theory (COB), that improves the information content of the joint space by minimizing the redundancy.
ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretization
Efficient detection and description of geometric regions in images is a prerequisite in visual systems for localization and mapping.
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features.
Ranked #5 on
Spectral Reconstruction
on Real HSI
Uncertainty-Aware Deep Multi-View Photometric Stereo
At each pixel, our approach either selects or discards deep-PS and deep-MVS network prediction depending on the prediction uncertainty measure.
Pix2NeRF: Unsupervised Conditional $π$-GAN for Single Image to Neural Radiance Fields Translation
We propose a pipeline to generate Neural Radiance Fields~(NeRF) of an object or a scene of a specific class, conditioned on a single input image.
Adiabatic Quantum Computing for Multi Object Tracking
Multi-Object Tracking (MOT) is most often approached in the tracking-by-detection paradigm, where object detections are associated through time.
Fast Online Video Super-Resolution with Deformable Attention Pyramid
Our DAP aligns and integrates information from the recurrent state into the current frame prediction.
VRT: A Video Restoration Transformer
Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping.
Ranked #1 on
Deblurring
on BASED
Revisiting RCAN: Improved Training for Image Super-Resolution
Image super-resolution (SR) is a fast-moving field with novel architectures attracting the spotlight.
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
In this work, we propose RePaint: A Denoising Diffusion Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
Collapse by Conditioning: Training Class-conditional GANs with Limited Data
On the contrary, we observe that class-conditioning causes mode collapse in limited data settings, where unconditional learning leads to satisfactory generative ability.
End-To-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization
In this work, we take a new route to learn to optimize the LiDAR beam configuration for a given application.
Flow-Guided Sparse Transformer for Video Deblurring
Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring.
Ranked #1 on
Deblurring
on DVD
Sound and Visual Representation Learning with Multiple Pretraining Tasks
Specifically, for this study, we investigate binaural sounds and image data in isolation.
Pix2NeRF: Unsupervised Conditional p-GAN for Single Image to Neural Radiance Fields Translation
We propose a pipeline to generate Neural Radiance Fields (NeRF) of an object or a scene of a specific class, conditioned on a single input image.
Facial-Sketch Synthesis: A New Challenge
Besides, we elaborate comprehensive experiments on the existing 19 cutting-edge models.
Improving the Behaviour of Vision Transformers with Token-consistent Stochastic Layers
We use linear layers with token-consistent stochastic parameters inside the multilayer perceptron blocks, without altering the architecture of the transformer.
End-to-End Learning of Multi-category 3D Pose and Shape Estimation
We use a Transformer-based architecture to detect the keypoints, as well as to summarize the visual context of the image.
Topology Preserving Local Road Network Estimation from Single Onboard Camera Image
We represent the road topology using a set of directed lane curves and their interactions, which are captured using their intersection points.
