Search Results for author:
Found 540 papers, 272 papers with code
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
In this work, we propose RePaint: A Denoising Diffusion Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
Exploring Cross-Image Pixel Contrast for Semantic Segmentation
Inspired by the recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive framework for semantic segmentation in the fully supervised setting.
SwinIR: Image Restoration Using Swin Transformer
In particular, the deep feature extraction module is composed of several residual Swin Transformer blocks (RSTB), each of which has several Swin Transformer layers together with a residual connection.
Ranked #2 on
Color Image Denoising
on urban100 sigma15
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
The HSI representations are highly similar and correlated across the spectral dimension.
Ranked #2 on
Spectral Reconstruction
on ARAD-1K
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction
Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).
Ranked #1 on
Spectral Reconstruction
on ARAD-1K
Temporal Segment Networks for Action Recognition in Videos
Furthermore, based on the temporal segment networks, we won the video classification track at the ActivityNet challenge 2016 among 24 teams, which demonstrates the effectiveness of TSN and the proposed good practices.
Ranked #5 on
Video Classification
on COIN
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
The other contribution is our study on a series of good practices in learning ConvNets on video data with the help of temporal segment network.
Ranked #3 on
Multimodal Activity Recognition
on EV-Action
Probabilistic Regression for Visual Tracking
In this work, we therefore propose a probabilistic regression formulation and apply it to tracking.
Ranked #4 on
Object Tracking
on FE108
Learning Discriminative Model Prediction for Tracking
The current strive towards end-to-end trainable computer vision systems imposes major challenges for the task of visual tracking.
Ranked #5 on
Object Tracking
on FE108
Know Your Surroundings: Exploiting Scene Information for Object Tracking
Such approaches are however prone to fail in case of e. g. fast appearance changes or presence of distractor objects, where a target appearance model alone is insufficient for robust tracking.
Ranked #8 on
Object Tracking
on FE108
Learning What to Learn for Video Object Segmentation
This allows us to achieve a rich internal representation of the target in the current frame, significantly increasing the segmentation accuracy of our approach.
Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videos
This effectively limits the performance and generalization capabilities of existing video segmentation methods.
Learning Target Candidate Association to Keep Track of What Not to Track
To tackle the problem of lacking ground-truth correspondences between distractor objects in visual tracking, we propose a training strategy that combines partial annotations with self-supervision.
Ranked #5 on
Visual Object Tracking
on OTB-2015
Efficient Visual Tracking with Exemplar Transformers
E. T. Track, our visual tracker that incorporates Exemplar Transformer modules, runs at 47 FPS on a CPU.
Transforming Model Prediction for Tracking
Optimization based tracking methods have been widely successful by integrating a target model prediction module, providing effective global reasoning by minimizing an objective function.
Ranked #21 on
Visual Object Tracking
on LaSOT
(Precision metric)
Robust Visual Tracking by Segmentation
We infer a bounding box from the segmentation mask, validate our tracker on challenging tracking datasets and achieve the new state of the art on LaSOT with a success AUC score of 69. 7%.
AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility
While being greatly benefiting to the tracking research, existing benchmarks do not pose the same difficulty as before with recent trackers achieving higher performance mainly due to (i) the introduction of more sophisticated transformers-based methods and (ii) the lack of diverse scenarios with adverse visibility such as, severe weather conditions, camouflage and imaging effects.
Towards End-to-End Lane Detection: an Instance Segmentation Approach
By doing so, we ensure a lane fitting which is robust against road plane changes, unlike existing approaches that rely on a fixed, pre-defined transformation.
Ranked #15 on
Lane Detection
on TuSimple
DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks
Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations - small sensor size, compact lenses and the lack of specific hardware, - impede them to achieve the quality results of DSLR cameras.
On the Practicality of Deterministic Epistemic Uncertainty
We find that, while DUMs scale to realistic vision tasks and perform well on OOD detection, the practicality of current methods is undermined by poor calibration under distributional shifts.
Out of Distribution (OOD) Detection
Semantic Segmentation
+1
SCAN: Learning to Classify Images without Labels
First, a self-supervised task from representation learning is employed to obtain semantically meaningful features.
Ranked #4 on
Unsupervised Image Classification
on ImageNet
Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations
Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection.
VRT: A Video Restoration Transformer
Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping.
Ranked #1 on
Deblurring
on BASED
Designing a Practical Degradation Model for Deep Blind Image Super-Resolution
It is widely acknowledged that single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images.
Deep Extreme Cut: From Extreme Points to Object Segmentation
This paper explores the use of extreme points in an object (left-most, right-most, top, bottom pixels) as input to obtain precise object segmentation for images and videos.
Deep Unfolding Network for Image Super-Resolution
As a result, the proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model, while maintaining the advantages of learning-based methods.
SRFlow: Learning the Super-Resolution Space with Normalizing Flow
SRFlow therefore directly accounts for the ill-posed nature of the problem, and learns to predict diverse photo-realistic high-resolution images.
Ranked #6 on
Image Super-Resolution
on DIV2K val - 4x upscaling
(using extra training data)
MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning
In this paper, we argue about the importance of considering task interactions at multiple scales when distilling task information in a multi-task learning setup.
Ranked #7 on
Semantic Segmentation
on UrbanLF
Multi-Task Learning for Dense Prediction Tasks: A Survey
In this survey, we provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision, explicitly emphasizing on dense prediction tasks.
End-to-end Lane Detection through Differentiable Least-Squares Fitting
The problem with such a two-step approach is that the parameters of the network are not optimized for the true task of interest (estimating the lane curvature parameters) but for a proxy task (segmenting the lane markings), resulting in sub-optimal performance.
Learning Accurate Dense Correspondences and When to Trust Them
Establishing dense correspondences between a pair of images is an important and general problem.
Warp Consistency for Unsupervised Learning of Dense Correspondences
From our observations and empirical results, we design a general unsupervised objective employing two of the derived constraints.
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
In order to apply dense methods to real-world applications, such as pose estimation, image manipulation, or 3D reconstruction, it is therefore crucial to estimate the confidence of the predicted matches.
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences
We propose Probabilistic Warp Consistency, a weakly-supervised learning objective for semantic matching.
Plug-and-Play Image Restoration with Deep Denoiser Prior
Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems.
Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis
While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved.
Ranked #1 on
Image Denoising
on urban100 sigma15
One-Shot Video Object Segmentation
This paper tackles the task of semi-supervised video object segmentation, i. e., the separation of an object from the background in a video, given the mask of the first frame.
Deep Temporal Linear Encoding Networks
Advantages of TLEs are: (a) they encode the entire video into a compact feature representation, learning the semantics and a discriminative feature space; (b) they are applicable to all kinds of networks like 2D and 3D CNNs for video classification; and (c) they model feature interactions in a more expressive way and without loss of information.
Generative Adversarial Networks for Extreme Learned Image Compression
We present a learned image compression system based on GANs, operating at extremely low bitrates.
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features.
Ranked #5 on
Spectral Reconstruction
on Real HSI
Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction
Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i. e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement.
Ranked #2 on
Spectral Reconstruction
on Real HSI
Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement.
Ranked #1 on
Spectral Reconstruction
on Real HSI
MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation
Estimating 3D human poses from monocular videos is a challenging task due to depth ambiguity and self-occlusion.
Ranked #22 on
3D Human Pose Estimation
on MPI-INF-3DHP
Sparse and noisy LiDAR completion with RGB guidance and uncertainty
However, we additionally propose a fusion method with RGB guidance from a monocular camera in order to leverage object information and to correct mistakes in the sparse input.
Ranked #5 on
Depth Completion
on KITTI Depth Completion
Sparse and noisy LiDAR completion with RGB guidance anduncertainty
For autonomous vehicles and robotics the use of LiDAR is indispensable in order to achieve precise depth predictions.
Semantic Instance Segmentation with a Discriminative Loss Function
In this work we propose to tackle the problem with a discriminative loss function, operating at the pixel level, that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step.
Ranked #4 on
Multi-Human Parsing
on MHP v1.0
DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation
It improves the state of the art by 10. 8 mIoU for GTA-to-Cityscapes and 5. 4 mIoU for Synthia-to-Cityscapes and enables learning even difficult classes such as train, bus, and truck well.
Ranked #5 on
Semantic Segmentation
on DensePASS
Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation
As previous UDA&DG semantic segmentation methods are mostly based on outdated networks, we benchmark more recent architectures, reveal the potential of Transformers, and design the DAFormer network tailored for UDA&DG.
Video Polyp Segmentation: A Deep Learning Perspective
We present the first comprehensive video polyp segmentation (VPS) study in the deep learning era.
Ranked #2 on
Video Polyp Segmentation
on SUN-SEG-Easy (Unseen)
Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals
To achieve this, we introduce a two-step framework that adopts a predetermined mid-level prior in a contrastive optimization objective to learn pixel embeddings.
Ranked #3 on
Unsupervised Semantic Segmentation
on ImageNet-S-50
Practical Full Resolution Learned Lossless Image Compression
We propose the first practical learned lossless image compression system, L3C, and show that it outperforms the popular engineered codecs, PNG, WebP and JPEG 2000.
Ranked #3 on
Image Compression
on ImageNet32
SEEDS: Superpixels Extracted via Energy-Driven Sampling
We define a robust and fast to evaluate energy function, based on enforcing color similarity between the bound- aries and the superpixel color histogram.
DiffIR: Efficient Diffusion Model for Image Restoration
Diffusion model (DM) has achieved SOTA performance by modeling the image synthesis process into a sequential application of a denoising network.
UniDepth: Universal Monocular Metric Depth Estimation
However, the remarkable accuracy of recent MMDE methods is confined to their training domains.
Ranked #2 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
Point-SLAM: Dense Neural Point Cloud-based SLAM
We propose a dense neural simultaneous localization and mapping (SLAM) approach for monocular RGBD input which anchors the features of a neural scene representation in a point cloud that is iteratively generated in an input-dependent data-driven manner.
Replacing Mobile Camera ISP with a Single Deep Learning Model
The model is trained to convert RAW Bayer data obtained directly from mobile camera sensor into photos captured with a professional high-end DSLR camera, making the solution independent of any particular mobile ISP implementation.
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
The aim of this paper is to propose a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration.
Ranked #1 on
Image Defocus Deblurring
on DPD (Dual-view)
Improving Video Generation for Multi-functional Applications
In this paper, we aim to improve the state-of-the-art video generative adversarial networks (GANs) with a view towards multi-functional applications.
Domain Adaptive Faster R-CNN for Object Detection in the Wild
The results demonstrate the effectiveness of our proposed approach for robust object detection in various domain shift scenarios.
Recurrent Video Restoration Transformer with Guided Deformable Attention
Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature.
Rethinking Semantic Segmentation: A Prototype View
Prevalent semantic segmentation solutions, despite their different network designs (FCN based or attention based) and mask decoding strategies (parametric softmax based or pixel-query based), can be placed in one category, by considering the softmax weights or query vectors as learnable class prototypes.
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
We then introduce a dual-branch Transformer-CNN feature extractor with Lite Transformer (LT) blocks leveraging long-range attention to handle low-frequency global features and Invertible Neural Networks (INN) blocks focusing on extracting high-frequency local information.
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM).
Equivariant Multi-Modality Image Fusion
These components enable the net training to follow the principles of the natural sensing-imaging process while satisfying the equivariant imaging prior.
Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation
Action segmentation refers to inferring boundaries of semantically consistent visual concepts in videos and is an important requirement for many video understanding tasks.
Ranked #1 on
Action Segmentation
on MPII Cooking 2 Dataset
Denoising Diffusion Models for Plug-and-Play Image Restoration
Although diffusion models have shown impressive performance for high-quality image synthesis, their potential to serve as a generative denoiser prior to the plug-and-play IR methods remains to be further explored.
Scribble-Supervised LiDAR Semantic Segmentation
Densely annotating LiDAR point clouds remains too expensive and time-consuming to keep up with the ever growing volume of data.
Ranked #2 on
3D Semantic Segmentation
on ScribbleKITTI
Pix2NeRF: Unsupervised Conditional $π$-GAN for Single Image to Neural Radiance Fields Translation
We propose a pipeline to generate Neural Radiance Fields~(NeRF) of an object or a scene of a specific class, conditioned on a single input image.
Pix2NeRF: Unsupervised Conditional p-GAN for Single Image to Neural Radiance Fields Translation
We propose a pipeline to generate Neural Radiance Fields (NeRF) of an object or a scene of a specific class, conditioned on a single input image.
CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016
This paper presents the method that underlies our submission to the untrimmed video classification task of ActivityNet Challenge 2016.
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach
Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense benchmark and state-of-the-art performance on the challenging public routes of the CARLA LeaderBoard.
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation
MIC significantly improves the state-of-the-art performance across the different recognition tasks for synthetic-to-real, day-to-nighttime, and clear-to-adverse-weather UDA.
Video Super-Resolution Transformer
Specifically, to tackle the first issue, we present a spatial-temporal convolutional self-attention layer with a theoretical understanding to exploit the locality information.
Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation
Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process.
Ranked #4 on
Semi-Supervised Semantic Segmentation
on Cityscapes 100 samples labeled
(using extra training data)
Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process.
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation
Therefore, we propose HRDA, a multi-resolution training approach for UDA, that combines the strengths of small high-resolution crops to preserve fine segmentation details and large low-resolution crops to capture long-range context dependencies with a learned scale attention, while maintaining a manageable GPU memory footprint.
Ranked #3 on
Semantic Segmentation
on GTAV-to-Cityscapes Labels
GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network
We propose GOCor, a fully differentiable dense matching module, acting as a direct replacement to the feature correlation layer.
Dense Pixel Correspondence Estimation
Feature Correlation
+2
Fast Scene Understanding for Autonomous Driving
Most approaches for instance-aware semantic labeling traditionally focus on accuracy.
Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth
In this work we propose a new clustering loss function for proposal-free instance segmentation.
Ranked #1000000000 on
Instance Segmentation
on Cityscapes test
Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement
In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames at encoder and decoder sides, respectively.
Perceptual Learned Video Compression with Recurrent Conditional GAN
This paper proposes a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional GAN.
Continual Test-Time Domain Adaptation
However, real-world machine perception systems are running in non-stationary and continually changing environments where the target domain distribution can change over time.
Appearance-and-Relation Networks for Video Classification
Specifically, SMART blocks decouple the spatiotemporal learning module into an appearance branch for spatial modeling and a relation branch for temporal modeling.
Ranked #51 on
Action Recognition
on UCF101
Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Images
In this work, we study the problem of extracting a directed graph representing the local road network in BEV coordinates, from a single onboard camera image.
Ranked #3 on
Lane Detection
on nuScenes
HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud
Extracting keypoint locations from input hand frames, known as 3D hand pose estimation, is a critical task in various human-computer interaction applications.
Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks
We present Convolutional Oriented Boundaries (COB), which produces multiscale oriented contours and region hierarchies starting from generic image classification Convolutional Neural Networks (CNNs).
Convolutional Oriented Boundaries
We present Convolutional Oriented Boundaries (COB), which produces multiscale oriented contours and region hierarchies starting from generic image classification Convolutional Neural Networks (CNNs).
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling
More specifically, HCFlow learns a bijective mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously.
Conditional Probability Models for Deep Image Compression
During training, the auto-encoder makes use of the context model to estimate the entropy of its representation, and the context model is concurrently updated to learn the dependencies between the symbols in the latent representation.
Dense 3D Regression for Hand Pose Estimation
Specifically, we decompose the pose parameters into a set of per-pixel estimations, i. e., 2D heat maps, 3D heat maps and unit 3D directional vector fields.
Ranked #4 on
Hand Pose Estimation
on MSRA Hands
A Survey on Deep Learning Technique for Video Segmentation
Video segmentation -- partitioning video frames into multiple segments or objects -- plays a critical role in a broad range of practical applications, from enhancing visual effects in movie, to understanding scenes in autonomous driving, to creating virtual background in video conferencing.
Deep Burst Super-Resolution
We propose a novel architecture for the burst super-resolution task.
Ranked #6 on
Burst Image Super-Resolution
on SyntheticBurst
Burst Image Super-Resolution
Multi-Frame Super-Resolution
+1
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising
The deep reparametrization allows us to directly model the image formation process in the latent space, and to integrate learned image priors into the prediction.
Ranked #4 on
Burst Image Super-Resolution
on BurstSR
GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector
In this paper, we present a novel end-to-end group collaborative learning network, termed GCoNet+, which can effectively and efficiently (250 fps) identify co-salient objects in natural scenes.
Ranked #1 on
Co-Salient Object Detection
on CoCA
Night-to-Day Image Translation for Retrieval-based Localization
We then compare the daytime and translated night images to obtain a pose estimate for the night image using the known 6-DOF position of the closest day image.
Disentangled Person Image Generation
Generating novel, yet realistic, images of persons is a challenging task due to the complex interplay between the different image factors, such as the foreground, background and pose information.
Ranked #2 on
Gesture-to-Gesture Translation
on Senz3D
Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution
Extensive experiments on synthetic and real images show that the proposed MANet not only performs favorably for both spatially variant and invariant kernel estimation, but also leads to state-of-the-art blind SR performance when combined with non-blind SR methods.
LiDAR Snowfall Simulation for Robust 3D Object Detection
Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds.
Ranked #1 on
3D Object Detection
on Heavy Snowfall
UntrimmedNets for Weakly Supervised Action Recognition and Detection
We exploit the learned models for action recognition (WSR) and detection (WSD) on the untrimmed video datasets of THUMOS14 and ActivityNet.
Ranked #3 on
Action Classification
on ActivityNet-1.2
Weakly Supervised Action Localization
Weakly-Supervised Action Recognition
Towards Partial Supervision for Generic Object Counting in Natural Scenes
Our RLC framework further reduces the annotation cost arising from large numbers of object categories in a dataset by only using lower-count supervision for a subset of categories and class-labels for the remaining ones.
Image Classification
Image-level Supervised Instance Segmentation
+3
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather
2) Through extensive experiments with several state-of-the-art detection approaches, we show that our fog simulation can be leveraged to significantly improve the performance for 3D object detection in the presence of fog.
Ranked #1 on
3D Object Detection
on Dense Fog
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation
Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge.
Flow-Guided Sparse Transformer for Video Deblurring
Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring.
Ranked #1 on
Deblurring
on DVD
Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration
On the other hand, we equip the sequence-to-sequence model with an unsupervised optical flow estimator to maximize its potential.
Ranked #2 on
Video Enhancement
on MFQE v2
Flow-based Kernel Prior with Application to Blind Super-Resolution
Kernel estimation is generally one of the key problems for blind image super-resolution (SR).
Dynamic Filter Networks
In a traditional convolutional layer, the learned filters stay fixed after training.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
ComboGAN: Unrestrained Scalability for Image Domain Translation
This year alone has seen unprecedented leaps in the area of learning-based image translation, namely CycleGAN, by Zhu et al.
Ranked #6 on
Facial Expression Translation
on CelebA
Knowledge Distillation based Degradation Estimation for Blind Super-Resolution
It consists of a knowledge distillation based implicit degradation estimator network (KD-IDE) and an efficient SR network.
AutoDecoding Latent 3D Diffusion Models
We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core.
Event-Based Fusion for Motion Deblurring with Cross-modal Attention
Traditional frame-based cameras inevitably suffer from motion blur due to long exposure times.
Ranked #3 on
Deblurring
on GoPro
(using extra training data)
Reference-based Image Super-Resolution with Deformable Attention Transformer
Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images.
Weakly Supervised 3D Object Detection from Lidar Point Cloud
This work proposes a weakly supervised approach for 3D object detection, only requiring a small set of weakly annotated scenes, associated with a few precisely labeled object instances.
P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior
We focus on the supervised setup, in which ground-truth depth is available only at training time.
Ranked #6 on
Depth Estimation
on NYU-Depth V2
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Basic Binary Convolution Unit for Binarized Image Restoration Network
In this study, we reconsider components in binary convolution, such as residual connection, BatchNorm, activation function, and structure, for IR tasks.
VA-DepthNet: A Variational Approach to Single Image Depth Prediction
While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene.
Ranked #18 on
Monocular Depth Estimation
on NYU-Depth V2
LocalViT: Bringing Locality to Vision Transformers
The importance of locality mechanisms is validated in two ways: 1) A wide range of design choices (activation function, layer placement, expansion ratio) are available for incorporating locality mechanisms and all proper choices can lead to a performance gain over the baseline, and 2) The same locality mechanism is successfully applied to 4 vision transformers, which shows the generalization of the locality concept.
Ranked #623 on
Image Classification
on ImageNet
Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification
Thus, by finetuning this network, we beat the performance of generic and recent methods in 3D CNNs, which were trained on large video datasets, e. g. Sports-1M, and finetuned on the target datasets, e. g. HMDB51/UCF101.
CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution
We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features.
SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation
Adapting to a continuously evolving environment is a safety-critical challenge inevitably faced by all autonomous driving systems.
Detection-aided liver lesion segmentation using deep learning
A fully automatic technique for segmenting the liver and localizing its unhealthy tissues is a convenient tool in order to diagnose hepatic diseases and assess the response to the according treatments.
Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding
The method is based on the fact that the results of semantic segmentation in moderately adverse conditions (light fog) can be bootstrapped to solve the same problem in highly adverse conditions (dense fog).
Ranked #5 on
Domain Adaptation
on Cityscapes-to-FoggyDriving
Video Object Segmentation with Episodic Graph Memory Networks
How to make a segmentation model efficiently adapt to a specific video and to online target appearance variations are fundamentally crucial issues in the field of video object segmentation.
Deep Gradient Learning for Efficient Camouflaged Object Detection
This paper introduces DGNet, a novel deep framework that exploits object gradient supervision for camouflaged object detection (COD).
Breathing New Life into 3D Assets with Generative Repainting
Diffusion-based text-to-image models ignited immense attention from the vision community, artists, and content creators.
DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows
We propose DeFlow, a method for learning stochastic image degradations from unpaired data.
Progressive Prioritized Multi-View Stereo
This work proposes a progressive patch based multi-view stereo algorithm able to deliver a dense point cloud at any time.
Covariance Pooling For Facial Expression Recognition
In this work, we explore the benefits of using a man- ifold network structure for covariance pooling to improve facial expression recognition.
Facial Expression Recognition
Facial Expression Recognition (FER)
Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image
Humans perceive the 3D world as a set of distinct objects that are characterized by various low-level (geometry, reflectance) and high-level (connectivity, adjacency, symmetry) properties.
Revisiting RCAN: Improved Training for Image Super-Resolution
Image super-resolution (SR) is a fast-moving field with novel architectures attracting the spotlight.
Structured Sparsity Learning for Efficient Video Super-Resolution
In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks.
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
The real-world deployment of an autonomous driving system requires its components to run on-board and in real-time, including the motion prediction module that predicts the future trajectories of surrounding traffic participants.
Depth Estimation from Monocular Images and Sparse Radar Data
We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations.
Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation
This paper presents MaskDistill: a novel framework for unsupervised semantic segmentation based on three key ideas.
Ranked #4 on
Unsupervised Semantic Segmentation
on PASCAL VOC 2012 val
(using extra training data)
Vision Transformers with Hierarchical Attention
This paper tackles the high computational/space complexity associated with Multi-Head Self-Attention (MHSA) in vanilla vision transformers.
OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers
We present OSFormer, the first one-stage transformer framework for camouflaged instance segmentation (CIS).
3D Appearance Super-Resolution with Deep Learning
Experimental results demonstrate that our proposed networks successfully incorporate the 3D geometric information and super-resolve the texture maps.
Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis
We propose a novel ECGAN for the challenging semantic image synthesis task.
Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis
To tackle 2), we design an effective module to selectively highlight class-dependent feature maps according to the original semantic layout to preserve the semantic information.
Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions
Due to the scarcity of dense pixel-level semantic annotations for images recorded in adverse visual conditions, there has been a keen interest in unsupervised domain adaptation (UDA) for the semantic segmentation of such images.
Ranked #1 on
Semantic Segmentation
on Dark Zurich
Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing
To address the challenging task of instance-aware human part parsing, a new bottom-up regime is proposed to learn category-level human semantic segmentation as well as multi-person pose estimation in a joint and end-to-end manner.
Learned Multi-Patch Similarity
Estimating a depth map from multiple views of a scene is a fundamental task in computer vision.
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
RayNet integrates a CNN that learns view-invariant feature representations with an MRF that explicitly encodes the physics of perspective projection and occlusion.
Learning Local and Global Temporal Contexts for Video Semantic Segmentation
Specifically, we uniformly sample certain frames from the video and extract global contextual prototypes by k-means.
SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance
In SemiVL, we propose to integrate rich priors from VLM pre-training into semi-supervised semantic segmentation to learn better semantic decision boundaries.
Ranked #1 on
Semi-Supervised Semantic Segmentation
on PASCAL VOC 2012 732 labeled
(using extra training data)
Understanding Bird's-Eye View of Road Semantics using an Onboard Camera
In this work, we study scene understanding in the form of online estimation of semantic BEV maps using the video input from a single onboard camera.
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
Large Scale Holistic Video Understanding
HVU is organized hierarchically in a semantic taxonomy that focuses on multi-label and multi-task video understanding as a comprehensive problem that encompasses the recognition of multiple semantic aspects in the dynamic scene.
Ranked #11 on
Action Recognition
on UCF101
Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
However, such a supervision is not always available.
Ranked #15 on
Domain Adaptation
on SYNTHIA-to-Cityscapes
(using extra training data)
Towards Interpretable Video Super-Resolution via Alternating Optimization
These issues can be alleviated by a cascade of three separate sub-tasks, including video deblurring, frame interpolation, and super-resolution, which, however, would fail to capture the spatial and temporal correlations among video sequences.
ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretization
Efficient detection and description of geometric regions in images is a prerequisite in visual systems for localization and mapping.
Learning to Prompt with Text Only Supervision for Vision-Language Models
While effective, most of these works require labeled data which is not practical, and often struggle to generalize towards new datasets due to over-fitting on the source data.
DLOW: Domain Flow for Adaptation and Generalization
In this work, we present a domain flow generation(DLOW) model to bridge two different domains by generating a continuous sequence of intermediate domains flowing from one domain to the other.
Topology Preserving Local Road Network Estimation from Single Onboard Camera Image
We represent the road topology using a set of directed lane curves and their interactions, which are captured using their intersection points.
Query-adaptive Video Summarization via Quality-aware Relevance Estimation
Although the problem of automatic video summarization has recently received a lot of attention, the problem of creating a video summary that also highlights elements relevant to a search query has been less studied.
Advances in Deep Concealed Scene Understanding
Concealed scene understanding (CSU) is a hot computer vision topic aiming to perceive objects exhibiting camouflage.
A Real-Time Online Learning Framework for Joint 3D Reconstruction and Semantic Segmentation of Indoor Scenes
This paper presents a real-time online vision framework to jointly recover an indoor scene's 3D structure and semantic label.
Semi-Supervised Learning by Augmented Distribution Alignment
We reveal that an essential sampling bias exists in semi-supervised learning due to the limited number of labeled samples, which often leads to a considerable empirical distribution mismatch between labeled data and unlabeled data.
AENet: Learning Deep Audio Features for Video Analysis
Instead, combining visual features with our AENet features, which can be computed efficiently on a GPU, leads to significant performance improvements on action recognition and video highlight detection.
Learning Filter Basis for Convolutional Neural Network Compression
Convolutional neural networks (CNNs) based solutions have achieved state-of-the-art performances for many computer vision tasks, including classification and super-resolution of images.
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
In this paper, we analyze two popular network compression techniques, i. e. filter pruning and low-rank decomposition, in a unified sense.
Talk2Car: Taking Control of Your Self-Driving Car
Or more specifically, we consider the problem in an autonomous driving setting, where a passenger requests an action that can be associated with an object found in a street scene.
SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
In contrast to previous methods that employ a discriminator that trivially concatenates semantics and image as an input, the SESAME discriminator is composed of two input streams that independently process the image and its semantics, using the latter to manipulate the results of the former.
Learning Better Lossless Compression Using Lossy Compression
We leverage the powerful lossy image compression algorithm BPG to build a lossless image compression system.
DHP: Differentiable Meta Pruning via HyperNetworks
Passing the sparsified latent vectors through the hypernetworks, the corresponding slices of the generated weight parameters can be removed, achieving the effect of network pruning.
Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning
We propose KYN, a novel method for single-view scene reconstruction that reasons about semantic and spatial context to predict each point's density.
Sliced Wasserstein Generative Models
In generative modeling, the Wasserstein distance (WD) has emerged as a useful metric to measure the discrepancy between generated and real data distributions.
Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs
Furthermore, we introduce a sliced version of Wasserstein GAN (SWGAN) loss to improve the distribution learning on the video data of high-dimension and mixed-spatiotemporal distribution.
Sliced Wasserstein Generative Models
In generative modeling, the Wasserstein distance (WD) has emerged as a useful metric to measure the discrepancy between generated and real data distributions.
Ranked #1 on
Video Generation
on TrailerFaces
Physical Adversarial Attack meets Computer Vision: A Decade Survey
Building upon this foundation, we uncover the pervasive role of artifacts carrying adversarial perturbations in the physical world.
Indiscernible Object Counting in Underwater Scenes
We further advance the frontier of this field by systematically studying a new challenge named indiscernible object counting (IOC), the goal of which is to count objects that are blended with respect to their surroundings.
TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles.
Deep Equilibrium Diffusion Restoration with Parallel Sampling
Due to the inherent property of diffusion models, most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs.
Semantic Understanding of Foggy Scenes with Purely Synthetic Data
This work addresses the problem of semantic scene understanding under foggy road conditions.
Efficient Conditional GAN Transfer with Knowledge Propagation across Classes
To address this problem, we introduce a new GAN transfer method to explicitly propagate the knowledge from the old classes to the new classes.
Facial-Sketch Synthesis: A New Challenge
Besides, we elaborate comprehensive experiments on the existing 19 cutting-edge models.
Exploring Relational Context for Multi-Task Dense Prediction
Our goal is to find the most efficient way to refine each task prediction by capturing cross-task contexts dependent on tasks' relations.
Ranked #78 on
Semantic Segmentation
on NYU Depth v2
Collapse by Conditioning: Training Class-conditional GANs with Limited Data
On the contrary, we observe that class-conditioning causes mode collapse in limited data settings, where unconditional learning leads to satisfactory generative ability.
Arbitrary-Scale Image Synthesis
Positional encodings have enabled recent works to train a single adversarial network that can generate images of different scales.
Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation
We investigate normal-to-adverse condition model adaptation for semantic segmentation, whereby image-level correspondences are available in the target domain.
Ranked #1 on
Source-Free Domain Adaptation
on Cityscapes to ACDC
Iterative Deep Learning for Road Topology Extraction
This paper tackles the task of estimating the topology of road networks from aerial images.
Event-Free Moving Object Segmentation from Moving Ego Vehicle
Subsequently, we devise EmoFormer, a novel network able to exploit the event data.
Single-Model and Any-Modality for Video Object Tracking
In practice, most existing RGB trackers learn a single set of parameters to use them across datasets and applications.
SMIT: Stochastic Multi-Label Image-to-Image Translation
This problem is highly challenging due to three main reasons: (i) unpaired datasets, (ii) multiple attributes, and (iii) the multimodality (e. g., style) associated with the translation.
Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation
Specifically, we show that: (1) our approach improves performance on all tasks when they are complementary and mutually dependent; (2) the CTRL helps to improve both semantic segmentation and depth estimation tasks performance in the challenging UDA setting; (3) the proposed ISL training scheme further improves the semantic segmentation performance.
Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model
We propose a novel framework, i. e., Predict, Prevent, and Evaluate (PPE), for disentangled text-driven image manipulation that requires little manual annotation while being applicable to a wide variety of manipulations.
Continuous Pose for Monocular Cameras in Neural Implicit Representation
In this paper, we showcase the effectiveness of optimizing monocular camera poses as a continuous function of time.
DeepProposals: Hunting Objects and Actions by Cascading Deep Convolutional Layers
In this paper, a new method for generating object and action proposals in images and videos is proposed.
DeepProposal: Hunting Objects by Cascading Deep Convolutional Layers
We generate hypotheses in a sliding-window fashion over different activation layers and show that the final convolutional layers can find the object of interest with high recall but poor localization due to the coarseness of the feature maps.
End-To-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization
In this work, we take a new route to learn to optimize the LiDAR beam configuration for a given application.
Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions
In this work, we propose to predict 3D eye gaze from weak supervision of eye semantic segmentation masks and direct supervision of a few 3D gaze vectors.
SMILE: Semantically-guided Multi-attribute Image and Layout Editing
Additionally, our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space, and it can be easily extended to head-swapping and face-reenactment applications without being trained on videos.
Global and Local Alignment Networks for Unpaired Image-to-Image Translation
The global alignment network aims to transfer the input image from the source domain to the target domain.
Masked Vision-Language Transformer in Fashion
We present a masked vision-language transformer (MVLT) for fashion-specific multi-modal representation.
Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference
First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning).
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials
Within the proposed benchmark, we explore some commonly known essentials of standard continual learning.
