Search Results for author:
Found 353 papers, 176 papers with code
Contextualized Spatio-Temporal Contrastive Learning with Self-Supervision
Modern self-supervised learning algorithms typically enforce persistency of instance representations across views.
VideoGLUE: Video General Understanding Evaluation of Foundation Models
We evaluate existing foundation models video understanding capabilities using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition, temporal localization, and spatiotemporal localization), eight datasets well received by the community, and four adaptation methods tailoring a foundation model (FM) for a downstream task.
Spatiotemporal Contrastive Video Representation Learning
Our representations are learned using a contrastive loss, where two augmented clips from the same short video are pulled together in the embedding space, while clips from different videos are pushed away.
Ranked #1 on
Self-Supervised Action Recognition
on Kinetics-600
COMISR: Compression-Informed Video Super-Resolution
Most video super-resolution methods focus on restoring high-resolution video frames from low-resolution videos without taking into account compression.
Video Timeline Modeling For News Story Understanding
In this paper, we present a novel problem, namely video timeline modeling.
Res2Net: A New Multi-scale Backbone Architecture
We evaluate the Res2Net block on all these models and demonstrate consistent performance gains over baseline models on widely-used datasets, e. g., CIFAR-100 and ImageNet.
Ranked #2 on
Image Classification
on GasHisSDB
A Closed-form Solution to Photorealistic Image Stylization
Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic.
Depth-Aware Video Frame Interpolation
The proposed model then warps the input frames, depth maps, and contextual features based on the optical flow and local interpolation kernels for synthesizing the output frame.
Ranked #5 on
Video Frame Interpolation
on Middlebury
Unified Visual Relationship Detection with Vision and Language Models
To address this challenge, we propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs).
Human-Object Interaction Detection
Relationship Detection
+2
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Finally, the two input images are warped and linearly fused to form each intermediate frame.
Diffusion Models: A Comprehensive Survey of Methods and Applications
This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration.
A Fusion Approach for Multi-Frame Optical Flow Estimation
To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account.
Learning Enriched Features for Real Image Restoration and Enhancement
With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing.
Ranked #5 on
Spectral Reconstruction
on ARAD-1K
CycleISP: Real Image Restoration via Improved Data Synthesis
This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline.
Ranked #10 on
Image Denoising
on DND
(using extra training data)
Multi-Stage Progressive Image Restoration
At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features.
Ranked #3 on
Spectral Reconstruction
on ARAD-1K
Restormer: Efficient Transformer for High-Resolution Image Restoration
Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks.
Ranked #1 on
Grayscale Image Denoising
on Urban100 sigma15
GAN Inversion: A Survey
GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model, for the image to be faithfully reconstructed from the inverted code by the generator.
Learning to See Through Obstructions
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions or raindrops, from a short sequence of images captured by a moving camera.
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
Learning to Adapt Structured Output Space for Semantic Segmentation
In this paper, we propose an adversarial learning method for domain adaptation in the context of semantic segmentation.
Ranked #3 on
Domain Adaptation
on Synscapes-to-Cityscapes
Adversarial Learning for Semi-Supervised Semantic Segmentation
We propose a method for semi-supervised semantic segmentation using an adversarial network.
Diverse Image-to-Image Translation via Disentangled Representations
Our model takes the encoded content features extracted from a given input and the attribute vectors sampled from the attribute space to produce diverse outputs at test time.
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
In this work, we present an approach based on disentangled representation for generating diverse outputs without paired training images.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation
To improve instance-level detection/segmentation performance, existing self-supervised and semi-supervised methods extract either task-unrelated or task-specific training signals from unlabeled data.
Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning
In the first stage, we obtain a robust feature extractor, which could serve for all images with base and novel categories.
AIMS: All-Inclusive Multi-Level Segmentation
Despite the progress of image segmentation for accurate visual entity segmentation, completing the diverse requirements of image editing applications for different-level region-of-interest selections remains unsolved.
High-Quality Entity Segmentation
It improves mask prediction by fusing high-res image crops that provide more fine-grained image details and the full image.
Rethinking Evaluation Metrics of Open-Vocabulary Segmentaion
We benchmarked the proposed evaluation metrics on 12 open-vocabulary methods of three segmentation tasks.
UniGS: Unified Representation for Image Generation and Segmentation
On the other hand, the progressive dichotomy module can efficiently decode the synthesized colormap to high-quality entity-level masks in a depth-first binary search without knowing the cluster numbers.
Universal Style Transfer via Feature Transforms
The whitening and coloring transforms reflect a direct matching of feature covariance of the content image to a given style image, which shares similar spirits with the optimization of Gram matrix based cost in neural style transfer.
Eidetic 3D LSTM: A Model for Video Prediction and Beyond
We first evaluate the E3D-LSTM network on widely-used future video prediction datasets and achieve the state-of-the-art performance.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
GLaMM: Pixel Grounding Large Multimodal Model
In this work, we present Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly intertwined with corresponding object segmentation masks.
Dancing to Music
In the analysis phase, we decompose a dance into a series of basic dance units, through which the model learns how to move.
Ranked #3 on
Motion Synthesis
on BRACE
Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline
We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization.
Hybrid Neural Fusion for Full-frame Video Stabilization
Existing video stabilization methods often generate visible distortion or require aggressive cropping of frame boundaries, resulting in smaller field of views.
Foundational Models Defining a New Era in Vision: A Survey and Outlook
Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world.
Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis
In this work, we propose a simple yet effective regularization term to address the mode collapse issue for cGANs.
Multimodal Unsupervised Image-To-Image Translation
Translation
Learning Blind Video Temporal Consistency
Our method takes the original unprocessed and per-frame processed videos as inputs to produce a temporally consistent video.
Learning Linear Transformations for Fast Arbitrary Style Transfer
Recent arbitrary style transfer methods transfer second order statistics from reference image onto content image via a multiplication between content image features and a transformation matrix, which is computed from features with a pre-determined algorithm.
Learning Enriched Features for Fast Image Restoration and Enhancement
In the former case, spatial details are preserved but the contextual information cannot be precisely encoded.
3D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field.
Superpixel Sampling Networks
Superpixels provide an efficient low/mid-level representation of image data, which greatly reduces the number of image primitives for subsequent vision tasks.
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Using our proposed efficient additive attention, we build a series of models called "SwiftFormer" which achieves state-of-the-art performance in terms of both accuracy and mobile inference speed.
Multi-Scale Boosted Dehazing Network with Dense Feature Fusion
To address the issue of preserving spatial information in the U-Net architecture, we design a dense feature fusion module using the back-projection feedback scheme.
Ranked #9 on
Image Dehazing
on Haze4k
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
Few-shot classification aims to recognize novel categories with only few labeled images in each class.
Ranked #6 on
Cross-Domain Few-Shot
on CUB
InfinityGAN: Towards Infinite-Pixel Image Synthesis
We present a novel framework, InfinityGAN, for arbitrary-sized image generation.
Ranked #2 on
Scene Generation
on OSM
Generative Face Completion
In this paper, we propose an effective face completion algorithm using a deep generative model.
ViDT: An Efficient and Effective Fully Transformer-based Object Detector
Transformers are transforming the landscape of computer vision, especially for recognition tasks.
Ranked #12 on
Object Detection
on COCO 2017 val
An Extendable, Efficient and Effective Transformer-based Object Detector
Transformers have been widely used in numerous vision problems especially for visual recognition and detection.
Class-agnostic Object Detection with Multi-modal Transformer
This has been a long-standing question in computer vision.
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation
Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks.
SPLATNet: Sparse Lattice Networks for Point Cloud Processing
We present a network architecture for processing point clouds that directly operates on a collection of points represented as a sparse set of samples in a high-dimensional lattice.
Ranked #30 on
Semantic Segmentation
on ScanNet
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer
In this paper, we investigate the application of Vehicle-to-Everything (V2X) communication to improve the perception performance of autonomous vehicles.
Ranked #1 on
3D Object Detection
on V2XSet
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
To the best of our knowledge, we are the first to try and solve the single-view reconstruction problem without a category-specific template mesh or semantic keypoints.
SCOPS: Self-Supervised Co-Part Segmentation
Parts provide a good intermediate representation of objects that is robust with respect to the camera, pose and appearance variations.
Ranked #4 on
Unsupervised Keypoint Estimation
on CUB
Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising
We present a spatial pixel aggregation network and learn the pixel sampling and averaging strategies for image denoising.
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks
However, existing methods often require a large number of network parameters and entail heavy computational loads at runtime for generating high-accuracy super-resolution results.
Decoupled Dynamic Filter Networks
Inspired by recent advances in attention, DDF decouples a depth-wise dynamic filter into spatial and channel dynamic filters.
Ranked #13 on
Semantic Segmentation
on MCubeS
A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence
Text-to-image diffusion models have made significant advances in generating and editing high-quality images.
Ranked #3 on
Semantic correspondence
on SPair-71k
RAP-SAM: Towards Real-Time All-Purpose Segment Anything
Segment Anything Model (SAM) is one remarkable model that can achieve generalized segmentation.
Self-regulating Prompts: Foundational Model Adaptation without Forgetting
To the best of our knowledge, this is the first regularization framework for prompt learning that avoids overfitting by jointly attending to pre-trained model features, the training trajectory during prompting, and the textual diversity.
Ranked #2 on
Prompt Engineering
on ImageNet V2
Collaborative Distillation for Ultra-Resolution Universal Style Transfer
In this work, we present a new knowledge distillation method (named Collaborative Distillation) for encoder-decoder based neural style transfer to reduce the convolutional filters.
Im2Pencil: Controllable Pencil Illustration from Photographs
We propose a high-quality photo-to-pencil translation method with fine-grained control over the drawing style.
SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
This paper proposes an end-to-end trainable network, SegFlow, for simultaneously predicting pixel-wise object segmentation and optical flow in videos.
Ranked #67 on
Semi-Supervised Video Object Segmentation
on DAVIS 2016
Weakly-Supervised Semantic Segmentation via Sub-category Exploration
Existing weakly-supervised semantic segmentation methods using image-level annotations typically rely on initial responses to locate object regions.
PiCANet: Learning Pixel-wise Contextual Attention for Saliency Detection
We formulate the proposed PiCANet in both global and local forms to attend to global and local contexts, respectively.
Ranked #7 on
RGB Salient Object Detection
on SOC
Switchable Temporal Propagation Network
Our approach is based on a temporal propagation network (TPN), which models the transition-related affinity between a pair of frames in a purely data-driven manner.
Joint-task Self-supervised Learning for Temporal Correspondence
Our learning process integrates two highly related tasks: tracking large image regions \emph{and} establishing fine-grained pixel-level associations between consecutive video frames.
Intriguing Properties of Vision Transformers
We show and analyze the following intriguing properties of ViT: (a) Transformers are highly robust to severe occlusions, perturbations and domain shifts, e. g., retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content.
MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement
Recently, a number of data-driven frame interpolation methods based on convolutional neural networks have been proposed.
Ranked #22 on
Video Frame Interpolation
on Vimeo90K
MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Frame Interpolation and Enhancement
In this work, we propose a motion estimation and motion compensation driven neural network for video frame interpolation.
Ranked #6 on
Video Frame Interpolation
on Middlebury
Regularizing Generative Adversarial Networks under Limited Data
Recent years have witnessed the rapid progress of generative adversarial networks (GANs).
Ranked #1 on
Image Generation
on CIFAR-100
Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector
A domain adaptive object detector aims to adapt itself to unseen domains that may contain variations of object appearance, viewpoints or backgrounds.
Large-scale Unsupervised Semantic Segmentation
In this work, we propose a new problem of large-scale unsupervised semantic segmentation (LUSS) with a newly created benchmark dataset to help the research progress.
Ranked #1 on
Unsupervised Semantic Segmentation
on ImageNet-S-300
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking
Second, we learn a correlation filter over a feature pyramid centered at the estimated target position for predicting scale changes.
Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking
Compared with SRDCF, STRCF with hand-crafted features provides a 5 times speedup and achieves a gain of 5. 4% and 3. 6% AUC score on OTB-2015 and Temple-Color, respectively.
Ranked #9 on
Visual Object Tracking
on VOT2017/18
MC-Blur: A Comprehensive Benchmark for Image Deblurring
Blur artifacts can seriously degrade the visual quality of images, and numerous deblurring methods have been proposed for specific scenarios.
Learning a No-Reference Quality Metric for Single-Image Super-Resolution
Numerous single-image super-resolution algorithms have been proposed in the literature, but few studies address the problem of performance evaluation based on visual perception.
Ranked #7 on
Video Quality Assessment
on MSU SR-QA Dataset
Gated Fusion Network for Joint Image Deblurring and Super-Resolution
Single-image super-resolution is a fundamental task for vision applications to enhance the image quality with respect to spatial resolution.
DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes
We present DrivingGaussian, an efficient and effective framework for surrounding dynamic autonomous driving scenes.
Burst Image Restoration and Enhancement
Our central idea is to create a set of pseudo-burst features that combine complementary information from all the input burst frames to seamlessly exchange information.
Ranked #2 on
Burst Image Super-Resolution
on BurstSR
Object Contour Detection with a Fully Convolutional Encoder-Decoder Network
We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network.
Progressive Domain Adaptation for Object Detection
This intermediate domain is constructed by translating the source images to mimic the ones in the target domain.
Burstormer: Burst Image Restoration and Enhancement Transformer
Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions.
Video Frame Interpolation Transformer
Existing methods for video interpolation heavily rely on deep convolution neural networks, and thus suffer from their intrinsic limitations, such as content-agnostic kernel weights and restricted receptive field.
Learning to Dub Movies via Hierarchical Prosody Models
Given a piece of text, a video clip and a reference audio, the movie dubbing (also known as visual voice clone V2C) task aims to generate speeches that match the speaker's emotion presented in the video using the desired speaker voice as reference.
Modeling Artistic Workflows for Image Generation and Editing
People often create art by following an artistic workflow involving multiple stages that inform the overall design.
Context-Aware Synthesis and Placement of Object Instances
Learning to insert an object instance into an image in a semantically coherent manner is a challenging and interesting problem.
Online Multi-Object Tracking with Dual Matching Attention Networks
In this paper, we propose an online Multi-Object Tracking (MOT) approach which integrates the merits of single object tracking and data association methods in a unified framework to handle noisy detections and frequent interactions between targets.
Ranked #5 on
Online Multi-Object Tracking
on MOT16
Learning to Localize Sound Sources in Visual Scenes: Analysis and Applications
Visual events are usually accompanied by sounds in our daily lives.
Progressive Representation Adaptation for Weakly Supervised Object Localization
In classification adaptation, we transfer a pre-trained network to a multi-label classification task for recognizing the presence of a certain object in an image.
Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation
In contrast to existing algorithms that tackle the tasks of semantic matching and object co-segmentation in isolation, our method exploits the complementary nature of the two tasks.
Learning to Caricature via Semantic Shape Transform
Caricature is an artistic drawing created to abstract or exaggerate facial features of a person.
Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection
Following this spirit, this paper explores plain ViT architecture for MUAD.
Flow-Grounded Spatial-Temporal Video Prediction from Still Images
Existing video prediction methods mainly rely on observing multiple historical frames or focus on predicting the next one-frame.
Deep Regression Tracking with Shrinkage Loss
Regression trackers directly learn a mapping from regularly dense samples of target objects to soft labels, which are usually generated by a Gaussian function, to estimate target positions.
Learning to See Through Obstructions with Layered Decomposition
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera.
Weakly-supervised Caricature Face Parsing through Domain Adaptation
However, current state-of-the-art face parsing methods require large amounts of labeled data on the pixel-level and such process for caricature is tedious and labor-intensive.
Hierarchical Modular Network for Video Captioning
(II) Predicate level, which learns the actions conditioned on highlighted objects and is supervised by the predicate in captions.
An Adaptive Random Path Selection Approach for Incremental Learning
In a conventional supervised learning setting, a machine learning model has access to examples of all object classes that are desired to be recognized during the inference stage.
Regularizing Meta-Learning via Gradient Dropout
With the growing attention on learning-to-learn new tasks using only a few examples, meta-learning has been widely used in numerous problems such as few-shot classification, reinforcement learning, and domain generalization.
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.
Learning to Blend Photos
Specifically, given a foreground image and a background image, our proposed method automatically generates a set of blending photos with scores that indicate the aesthetics quality with the proposed quality network and policy network.
Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training
Generating images from graph-structured inputs, such as scene graphs, is uniquely challenging due to the difficulty of aligning nodes and connections in graphs with objects and their relations in images.
Unsupervised Representation Learning by Sorting Sequences
We present an unsupervised representation learning approach using videos without semantic labels.
Ranked #46 on
Self-Supervised Action Recognition
on HMDB51
Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors
Generating a smooth sequence of intermediate results bridges the gap of two different domains, facilitating the morphing effect across domains.
Exploiting Raw Images for Real-Scene Super-Resolution
In this paper, we study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images.
Quadratic video interpolation
Video interpolation is an important problem in computer vision, which helps overcome the temporal limitation of camera sensors.
Efficient Video Object Segmentation via Modulated Cross-Attention Memory
Recently, transformer-based approaches have shown promising results for semi-supervised video object segmentation.
Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks
The proposed network is composed of three deep convolutional neural networks (CNNs) and a recurrent neural network (RNN).
Ranked #10 on
Deblurring
on RealBlur-R (trained on GoPro)
(SSIM (sRGB) metric)
Learning to Stylize Novel Views
Second, we develop point cloud aggregation modules to gather the style information of the 3D scene, and then modulate the features in the point cloud with a linear transformation matrix.
Learning Visibility for Robust Dense Human Body Estimation
An alternative approach is to estimate dense vertices of a predefined template body in the image space.
Simultaneous Fidelity and Regularization Learning for Image Restoration
For blind deconvolution, as estimation error of blur kernel is usually introduced, the subsequent non-blind deconvolution process does not restore the latent image well.
PiCANet: Pixel-wise Contextual Attention Learning for Accurate Saliency Detection
We propose three specific formulations of the PiCANet via embedding the pixel-wise contextual attention mechanism into the pooling and convolution operations with attending to global or local contexts.
Exploiting Diffusion Prior for Generalizable Dense Prediction
Contents generated by recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate due to the immitigable domain gap.
Tracking Persons-of-Interest via Unsupervised Representation Adaptation
Multi-face tracking in unconstrained videos is a challenging problem as faces of one person often appear drastically different in multiple shots due to significant variations in scale, pose, expression, illumination, and make-up.
Robust Visual Tracking via Hierarchical Convolutional Features
Specifically, we learn adaptive correlation filters on the outputs from each convolutional layer to encode the target appearance.
BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios
Based on the property of outdoor point clouds in autonomous driving scenarios, i. e., the point clouds of distant objects are more sparse, we propose point density prediction to enable the 3D encoder to learn location information, which is essential for object detection.
Text-Driven Image Editing via Learnable Regions
Language has emerged as a natural interface for image editing.
Referring Expression Object Segmentation with Caption-Aware Consistency
To this end, we propose an end-to-end trainable comprehension network that consists of the language and visual encoders to extract feature representations from both domains.
Ranked #19 on
Referring Expression Segmentation
on RefCOCO testB
Gated Fusion Network for Degraded Image Super Resolution
To address this problem, we propose a dual-branch convolutional neural network to extract base features and recovered features separately.
Autoregressive 3D Shape Generation via Canonical Mapping
With the capacity of modeling long-range dependencies in sequential data, transformers have shown remarkable performances in a variety of generative tasks such as image, audio, and text generation.
CiteTracker: Correlating Image and Text for Visual Tracking
Existing visual tracking methods typically take an image patch as the reference of the target to perform tracking.
Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence
This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing.
Ranked #1 on
Semantic correspondence
on PF-PASCAL
Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection
Semi-supervised object detection is crucial for 3D scene understanding, efficiently addressing the limitation of acquiring large-scale 3D bounding box annotations.
PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection
Recent temporal LiDAR-based 3D object detectors achieve promising performance based on the two-stage proposal-based approach.
Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration
All-in-one image restoration tackles different types of degradations with a unified model instead of having task-specific, non-generic models for each degradation.
Pyramid Diffusion for Fine 3D Large Scene Generation
Directly transferring the 2D techniques to 3D scene generation is challenging due to significant resolution reduction and the scarcity of comprehensive real-world 3D scene datasets.
Improving Zero-shot Generalization and Robustness of Multi-modal Models
We also show that our method improves across ImageNet shifted datasets, four other datasets, and other model architectures such as LiT.
Self-Supervised Super-Plane for Neural 3D Reconstruction
In this work, we introduce a self-supervised super-plane constraint by exploring the free geometry cues from the predicted surface, which can further regularize the reconstruction of plane regions without any other ground truth annotations.
CLR: Channel-wise Lightweight Reprogramming for Continual Learning
Reprogramming parameters are task-specific and exclusive to each task, which makes our method immune to catastrophic forgetting.
Superpixel Hierarchy
Quantitative and qualitative evaluation on a number of computer vision applications was conducted, demonstrating that the proposed method is the top performer.
Delving into Motion-Aware Matching for Monocular 3D Object Tracking
In this paper, we find that the motion cue of objects along different time frames is critical in 3D multi-object tracking, which is less explored in existing monocular-based approaches.
Correlation Tracking via Joint Discrimination and Reliability Learning
To address this issue, we propose a novel CF-based optimization problem to jointly model the discrimination and reliability information.
Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective
Object frequency in the real world often follows a power law, leading to a mismatch between datasets with long-tailed class distributions seen by a machine learning model and our expectation of the model to perform well on all classes.
Ranked #27 on
Long-tail Learning
on Places-LT
TapLab: A Fast Framework for Semantic Video Segmentation Tapping into Compressed-Domain Knowledge
Real-time semantic video segmentation is a challenging task due to the strict requirements of inference speed.
Scene Parsing with Global Context Embedding
We present a scene parsing method that utilizes global context information based on both the parametric and non- parametric models.
MiniROAD: Minimal RNN Framework for Online Action Detection
Online Action Detection (OAD) is the task of identifying actions in streaming videos without access to future frames.
Ranked #1 on
Online Action Detection
on TVSeries
Integrating Boundary and Center Correlation Filters for Visual Tracking with Aspect Ratio Variation
The aspect ratio variation frequently appears in visual tracking and has a severe influence on performance.
HENet: Hybrid Encoding for End-to-end Multi-task 3D Perception from Multi-view Cameras
Three-dimensional perception from multi-view cameras is a crucial component in autonomous driving systems, which involves multiple tasks like 3D object detection and bird's-eye-view (BEV) semantic segmentation.
End-to-end Multi-modal Video Temporal Grounding
Specifically, we adopt RGB images for appearance, optical flow for motion, and depth maps for image structure.
Generative Multiplane Neural Radiance for 3D-Aware Image Generation
We present a method to efficiently generate 3D-aware high-resolution images that are view-consistent across multiple target views.
The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation
Vision-and-Language Navigation (VLN) requires an agent to find a path to a remote location on the basis of natural-language instructions and a set of photo-realistic panoramas.
Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble
Automatically estimating 3D skeleton, shape, camera viewpoints, and part articulation from sparse in-the-wild image ensembles is a severely under-constrained and challenging problem.
Dual Associated Encoder for Face Restoration
Restoring facial details from low-quality (LQ) images has remained a challenging problem due to its ill-posedness induced by various degradations in the wild.
Ranked #2 on
Blind Face Restoration
on WIDER
An Informative Tracking Benchmark
Along with the rapid progress of visual tracking, existing benchmarks become less informative due to redundancy of samples and weak discrimination between current trackers, making evaluations on all datasets extremely time-consuming.
Learning Spatial-Aware Regressions for Visual Tracking
Second, we propose a fully convolutional neural network with spatially regularized kernels, through which the filter kernel corresponding to each output channel is forced to focus on a specific region of the target.
Ranked #12 on
Visual Object Tracking
on VOT2017/18
Dual Deep Network for Visual Tracking
In this paper, we propose a dual network to better utilize features among layers for visual tracking.
2.5D Visual Relationship Detection
To enable progress on this task, we create a new dataset consisting of 220k human-annotated 2. 5D relationships among 512K objects from 11K images.
FlowNAS: Neural Architecture Search for Optical Flow Estimation
To address this issue, we propose a neural architecture search method named FlowNAS to automatically find the better encoder architecture for flow estimation task.
Weakly Supervised Coupled Networks for Visual Sentiment Analysis
The second branch utilizes both the holistic and localized information by coupling the sentiment map with deep features for robust classification.
Unsupervised Holistic Image Generation from Key Local Patches
We introduce a new problem of generating an image based on a small number of key local patches without any geometric prior.
Semi-Supervised Learning with Meta-Gradient
In this work, we propose a simple yet effective meta-learning algorithm in semi-supervised learning.
PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal
For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts.
ReMix: Towards Image-to-Image Translation with Limited Data
We interpolate training samples at the feature level and propose a novel content loss based on the perceptual relations among samples.
Learning Object-level Point Augmentor for Semi-supervised 3D Object Detection
In this work, we propose an object-level point augmentor (OPA) that performs local transformations for semi-supervised 3D object detection.
D2-Net: Weakly-Supervised Action Localization via Discriminative Embeddings and Denoised Activations
The proposed formulation comprises a discriminative and a denoising loss term for enhancing temporal action localization.
Ranked #3 on
Weakly Supervised Action Localization
on THUMOS’14
Understanding Synonymous Referring Expressions via Contrastive Features
While prior work usually treats each sentence and attends it to an object separately, we focus on learning a referring expression comprehension model that considers the property in synonymous sentences.
RTracker: Recoverable Tracking via PN Tree Structured Memory
Upon the PN tree memory, we develop corresponding walking rules for determining the state of the target and define a set of control flows to unite the tracker and the detector in different tracking scenarios.
Progressive Multi-resolution Loss for Crowd Counting
Crowd counting is usually handled in a density map regression fashion, which is supervised via a L2 loss between the predicted density map and ground truth.
Online multi-object tracking via robust collaborative model and sample selection
For each frame, we construct an association between detections and trackers, and treat each detected image region as a key sample, for online update, if it is associated to a tracker.
Ranked #1 on
Online Multi-Object Tracking
on Oxford Town Center
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Convolutional neural networks have recently demonstrated high-quality reconstruction for single-image super-resolution.
Ranked #40 on
Image Super-Resolution
on BSD100 - 4x upscaling
Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection
To address these limitations, we propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection, which learns multi-scale temporal features.
Exploring Cross-Video and Cross-Modality Signals for Weakly-Supervised Audio-Visual Video Parsing
The audio-visual video parsing task aims to temporally parse a video into audio or visual event categories.
Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle
Recent advances in convolutional neural networks(CNNs) usually come with the expense of excessive computational overhead and memory footprint.
Learnable Cost Volume Using the Cayley Representation
Cost volume is an essential component of recent deep models for optical flow estimation and is usually constructed by calculating the inner product between two feature vectors.
Learning Discriminative Shrinkage Deep Networks for Image Deconvolution
Most existing methods usually formulate the non-blind deconvolution problem into a maximum-a-posteriori framework and address it by manually designing kinds of regularization terms and data terms of the latent clear images.
Towards 4D Human Video Stylization
To overcome these limitations, we leverage Neural Radiance Fields (NeRFs) to represent videos, conducting stylization in the rendered feature space.
Parallel Coordinate Descent Newton Method for Efficient $\ell_1$-Regularized Minimization
We show that: (1) PCDN is guaranteed to converge globally despite increasing parallelism; (2) PCDN converges to the specified accuracy $\epsilon$ within the limited iteration number of $T_\epsilon$, and $T_\epsilon$ decreases with increasing parallelism (bundle size $P$).
Unsupervised Discovery of Disentangled Manifolds in GANs
Interpretable generation process is beneficial to various image editing applications.
Training Class-Imbalanced Diffusion Model Via Overlap Optimization
To address the observed appearance overlap between synthesized images of rare classes and tail classes, we propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting
% To provide a more realistic reflection of the underlying practical challenge, we introduce a weakly supervised VIC task, wherein trajectory labels are not provided.
Learning to Deblur Images with Exemplars
However, existing methods are less effective as only few edges can be restored from blurry face images for kernel estimation.
Learning Dual Convolutional Neural Networks for Low-Level Vision
These problems usually involve the estimation of two components of the target signals: structures and details.
Visual Tracking via Dynamic Graph Learning
First, the graph is initialized by assigning binary weights of some image patches to indicate the object and background patches according to the predicted bounding box.
VITAL: VIsual Tracking via Adversarial Learning
To augment positive samples, we use a generative network to randomly generate masks, which are applied to adaptively dropout input features to capture a variety of appearance changes.
Learning a Discriminative Prior for Blind Image Deblurring
We present an effective blind image deblurring method based on a data-driven discriminative prior. Our work is motivated by the fact that a good image prior should favor clear images over blurred images. In this work, we formulate the image prior as a binary classifier which can be achieved by a deep convolutional neural network (CNN). The learned prior is able to distinguish whether an input image is clear or not. Embedded into the maximum a posterior (MAP) framework, it helps blind deblurring in various scenarios, including natural, face, text, and low-illumination images. However, it is difficult to optimize the deblurring method with the learned image prior as it involves a non-linear CNN. Therefore, we develop an efficient numerical approach based on the half-quadratic splitting method and gradient decent algorithm to solve the proposed model. Furthermore, the proposed model can be easily extended to non-uniform deblurring. Both qualitative and quantitative experimental results show that our method performs favorably against state-of-the-art algorithms as well as domain-specific image deblurring approaches.
Gated Fusion Network for Single Image Dehazing
The proposed algorithm hinges on an end-to-end trainable neural network that consists of an encoder and a decoder.
Ranked #23 on
Image Dehazing
on SOTS Outdoor
Deep Semantic Face Deblurring
In this paper, we present an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks (CNNs).
Learning to Localize Sound Source in Visual Scenes
We show that even with a few supervision, false conclusion is able to be corrected and the source of sound in a visual scene can be localized effectively.
Learning Video-Story Composition via Recurrent Neural Network
In this paper, we propose a learning-based method to compose a video-story from a group of video clips that describe an activity or experience.
Generative Single Image Reflection Separation
Single image reflection separation is an ill-posed problem since two scenes, a transmitted scene and a reflected scene, need to be inferred from a single observation.
Learning Binary Residual Representations for Domain-specific Video Streaming
Specifically, we target a streaming setting where the videos to be streamed from a server to a client are all in the same domain and they have to be compressed to a small size for low-latency transmission.
Joint Image Filtering with Deep Convolutional Networks
In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images.
Semantic-driven Generation of Hyperlapse from $360^\circ$ Video
We present a system for converting a fully panoramic ($360^\circ$) video into a normal field-of-view (NFOV) hyperlapse for an optimal viewing experience.
Learning Affinity via Spatial Propagation Networks
Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be the output from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix.
Learning to Segment Instances in Videos with Spatial Propagation Network
In addition, we apply a filter on the refined score map that aims to recognize the best connected region using spatial and temporal consistencies in the video.
Stylizing Face Images via Multiple Exemplars
We address the problem of transferring the style of a headshot photo to face images.
Video Deblurring via Semantic Segmentation and Pixel-Wise Non-Linear Kernel
We analyze the relationship between motion blur trajectory and optical flow, and present a novel pixel-wise non-linear kernel model to account for motion blur.
Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Despite rapid advances in face recognition, there remains a clear gap between the performance of still image-based face recognition and video-based face recognition, due to the vast difference in visual quality between the domains and the difficulty of curating diverse large-scale video datasets.
Face Parsing via Recurrent Propagation
Face parsing is an important problem in computer vision that finds numerous applications including recognition and editing.
CREST: Convolutional Residual Learning for Visual Tracking
Our method integrates feature extraction, response map generation as well as model update into the neural networks for an end-to-end training.
Learning Structured Semantic Embeddings for Visual Recognition
First, we exploit the discriminative constraints to capture the intra- and inter-class relationships of image embeddings.
Diversified Texture Synthesis with Feed-forward Networks
Recent progresses on deep discriminative and generative modeling have shown promising results on texture synthesis.
Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution
In this paper, we propose a fully convolutional networks for iterative non-blind deconvolution We decompose the non-blind deconvolution problem into image denoising and image deconvolution.
Visual Tracking via Boolean Map Representations
In this paper, we present a simple yet effective Boolean map based representation that exploits connectivity cues for visual tracking.
