Search Results for author:
Found 182 papers, 58 papers with code
Global-and-Local Relative Position Embedding for Unsupervised Video Summarization
In order to summarize a content video properly, it is important to grasp the sequential structure of video as well as the long-term dependency between frames.
Stable Surface Regularization for Fast Few-Shot NeRF
This paper proposes an algorithm for synthesizing novel views under few-shot setup.
MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark
Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras.
Towards Understanding Dual BN In Hybrid Adversarial Training
There is a growing concern about applying batch normalization (BN) in adversarial training (AT), especially when the model is trained on both adversarial samples and clean samples (termed Hybrid-AT).
ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object
In this work, we introduce generative model as a data source for synthesizing hard images that benchmark deep models' robustness.
DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolution
It is well known the adversarial optimization of GAN-based image super-resolution (SR) methods makes the preceding SR model generate unpleasant and undesirable artifacts, leading to large distortion.
Blurry Video Compression: A Trade-off between Visual Enhancement and Data Compression
Existing video compression (VC) methods primarily aim to reduce the spatial and temporal redundancies between consecutive frames in a video while preserving its quality.
MoDA: Leveraging Motion Priors from Videos for Advancing Unsupervised Domain Adaptation in Semantic Segmentation
Then, we propose a semantic mining module that takes the object masks to refine the pseudo labels in the target domain.
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge.
Long-range Multimodal Pretraining for Movie Understanding
While great progress has been attained, there is still a need for a pretrained multimodal model that can perform well in the ever-growing set of movie understanding tasks the community has been establishing.
ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution
To speed up inference and further enhance the performance, our research revisits diffusion models in image super-resolution and proposes a straightforward yet significant diffusion model-based super-resolution method called ACDMSR (accelerated conditional diffusion model for image super-resolution).
Learning from Multi-Perception Features for Real-Word Image Super-resolution
Currently, there are two popular approaches for addressing real-world image super-resolution problems: degradation-estimation-based and blind-based methods.
Attack-SAM: Towards Attacking Segment Anything Model With Adversarial Examples
Beyond the basic goal of mask removal, we further investigate and find that it is possible to generate any desired mask by the adversarial attack.
Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation
The meta architecture of the proposed Video-kMaX consists of two components: within clip segmenter (for clip-level segmentation) and cross-clip associater (for association beyond clips).
One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Era
Overall, this work is the first to survey ChatGPT with a comprehensive review of its underlying technology, applications, and challenges.
Complementary Random Masking for RGB-Thermal Semantic Segmentation
Also, the proposed self-distillation loss encourages the network to extract complementary and meaningful representations from a single modality or complementary masked modalities.
Ranked #2 on
Thermal Image Segmentation
on MFN Dataset
Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples
The objective of this work is to explore the learning of visually grounded speech models (VGS) from multilingual perspective.
TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation
Unlike previous unsupervised domain adaptation methods for category-level object pose estimation, our approach processes the test data in a sequential, online manner, and it does not require access to the source domain at runtime.
A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI
This work conducts a survey on audio diffusion model, which is complementary to existing surveys that either lack the recent progress of diffusion-based speech synthesis or highlight an overall picture of applying diffusion model in multiple fields.
A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?
As ChatGPT goes viral, generative AI (AIGC, a. k. a AI-generated content) has made headlines everywhere because of its ability to analyze and create text, images, and beyond.
Self-Sufficient Framework for Continuous Sign Language Recognition
The goal of this work is to develop self-sufficient framework for Continuous Sign Language Recognition (CSLR) that addresses key issues of sign language recognition.
Text-to-image Diffusion Models in Generative AI: A Survey
This survey reviews text-to-image diffusion models in the context that diffusion models have emerged to be popular for a wide range of generative tasks.
EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization
Second, our novel self-distilled regularization controls the output of the meta networks not to deviate significantly from the output of the frozen original networks, thereby preserving well-trained knowledge from the source domain.
CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution
Diffusion probabilistic models (DPM) have been widely adopted in image-to-image translation to generate high-quality images.
Semi-Supervised Image Captioning by Adversarially Propagating Labeled Data
We present a novel data-efficient semi-supervised framework to improve the generalization of image captioning models.
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition benchmarks, including ImageNet classification, COCO detection, and ADE20K segmentation.
Ranked #45 on
Semantic Segmentation
on ADE20K
Deep Depth Estimation From Thermal Image
Secondly, we conduct an exhaustive validation process of monocular and stereo depth estimation algorithms designed on visible spectrum bands to benchmark their performance in the thermal image domain.
Single View Scene Scale Estimation Using Scale Field
In this paper, we propose a single image scale estimation method based on a novel scale field representation.
Spacetime Surface Regularization for Neural Dynamic Scene Reconstruction
We propose an algorithm, 4DRegSDF, for the spacetime surface regularization to improve the fidelity of neural rendering and reconstruction in dynamic scenes.
Mask-Guided Matting in the Wild
Mask-guided matting has shown great practicality compared to traditional trimap-based methods.
Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection
First, no tracking supervisions are in LVIS, which leads to inconsistent learning of detection (with LVIS and TAO) and tracking (only with TAO).
Tracking by Associating Clips
The tracking-by-detection paradigm today has become the dominant method for multi-object tracking and works by detecting objects in each frame and then performing data association across frames.
Learning Classifiers of Prototypes and Reciprocal Points for Universal Domain Adaptation
To successfully train our framework, we collect the partial, confident target samples that are classified as known or unknown through on our proposed multi-criteria selection.
Test-time Adaptation in the Dynamic World with Compound Domain Knowledge Management
In addition, to prevent overfitting of the TTA model, we devise novel regularization which modulates the adaptation rates using domain-similarity between the source and the current target domain.
MATE: Masked Autoencoders are Online 3D Test-Time Learners
Our MATE is the first Test-Time-Training (TTT) method designed for 3D data, which makes deep networks trained for point cloud classification robust to distribution shifts occurring in test data.
Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition
Most existing Continuous Sign Language Recognition (CSLR) benchmarks have fixed backgrounds and are filmed in studios with a static monochromatic background.
One-Shot Neural Fields for 3D Object Understanding
At test-time, we build the representation from a single RGB input image observing the scene from only one viewpoint.
Moving from 2D to 3D: volumetric medical image classification for rectal cancer staging
In this study, we present a volumetric convolutional neural network to accurately discriminate T2 from T3 stage rectal cancer with rectal MR volumes.
Per-Clip Video Object Segmentation
In this per-clip inference scheme, we update the memory with an interval and simultaneously process a set of consecutive frames (i. e. clip) between the memory updates.
Generative Bias for Robust Visual Question Answering
In this work, in order to better learn the bias a target VQA model suffers from, we propose a generative method to train the bias model directly from the target model, called GenB.
A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond
Masked autoencoders are scalable vision learners, as the title of MAE \cite{he2022masked}, which suggests that self-supervised learning (SSL) in vision might undertake a similar trajectory as in NLP.
Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness
Adversarial training (AT) for robust representation learning and self-supervised learning (SSL) for unsupervised representation learning are two active research fields.
The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing
Machine learning is transforming the video editing industry.
ML-BPM: Multi-teacher Learning with Bidirectional Photometric Mixing for Open Compound Domain Adaptation in Semantic Segmentation
Open compound domain adaptation (OCDA) considers the target domain as the compound of multiple unknown homogeneous subdomains.
DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning
In this paper, we propose a multi-objective camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist of network-based and conventional ISP tools.
Labeling Where Adapting Fails: Cross-Domain Semantic Segmentation with Point Supervision via Active Selection
In this work, we propose a new domain adaptation framework for semantic segmentation with annotated points via active selection.
TubeFormer-DeepLab: Video Mask Transformer
We present TubeFormer-DeepLab, the first attempt to tackle multiple core video segmentation tasks in a unified manner.
MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation
In this paper, we propose and explore a new multi-modal extension of test-time adaptation for 3D semantic segmentation.
How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning
This yields a unified perspective on how negative samples and SimSiam alleviate collapse.
Investigating Top-$k$ White-Box and Transferable Black-box Attack
It is widely reported that stronger I-FGSM transfers worse than simple FGSM, leading to a popular belief that transferability is at odds with the white-box attack strength.
Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo
Contrastive learning (CL) is widely known to require many negative samples, 65536 in MoCo for instance, for which the performance of a dictionary-free framework is often inferior because the negative sample size (NSS) is limited by its mini-batch size (MBS).
Long-term Video Frame Interpolation via Feature Propagation
We argue that when there is a large gap between inputs, instead of estimating imprecise motion that will eventually lead to inaccurate interpolation, we can safely propagate from one side of the input up to a reliable time frame using the other input as a reference.
Audio-Visual Fusion Layers for Event Type Aware Video Recognition
Human brain is continuously inundated with the multisensory information and their complex interactions coming from the outside world at any given moment.
Fast Adversarial Training with Noise Augmentation: A Unified Perspective on RandStart and GradAlign
The former works only for a relatively small perturbation 8/255 with the l_\infty constraint, and GradAlign improves it by extending the perturbation size to 16/255 (with the l_\infty constraint) but at the cost of being 3 to 4 times slower.
Learning Sound Localization Better From Semantically Similar Samples
Thus, these semantically correlated pairs, "hard positives", are mistakenly grouped as negatives.
MC-Calib: A generic and robust calibration toolbox for multi-camera systems
In this paper, we present MC-Calib, a novel and robust toolbox dedicated to the calibration of complex synchronized multi-camera systems using an arbitrary number of fiducial marker-based patterns.
Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion
Based on the analysis, we propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency.
Investigating Top-k White-Box and Transferable Black-Box Attack
It is widely reported that stronger I-FGSM transfers worse than simple FGSM, leading to a popular belief that transferability is at odds with the white-box attack strength.
Facial Depth and Normal Estimation using Single Dual-Pixel Camera
Many mobile manufacturers recently have adopted Dual-Pixel (DP) sensors in their flagship models for faster auto-focus and aesthetic image captures.
UDA-COPE: Unsupervised Domain Adaptation for Category-level Object Pose Estimation
Inspired by recent multi-modal UDA techniques, the proposed method exploits a teacher-student self-supervised learning scheme to train a pose estimation network without using target domain pose labels.
Ranked #5 on
6D Pose Estimation using RGBD
on REAL275
Deep Point Cloud Reconstruction
In particular, we further improve the performance of transformer by a newly proposed module called amplified positional encoding.
PointMixer: MLP-Mixer for Point Cloud Understanding
MLP-Mixer has newly appeared as a new challenger against the realm of CNNs and transformer.
Ranked #19 on
Semantic Segmentation
on S3DIS Area5
Self-Supervised Real-time Video Stabilization
Videos are a popular media form, where online video streaming has recently gathered much popularity.
Single-Modal Entropy based Active Learning for Visual Question Answering
In this work, we address Active Learning in the multi-modal setting of Visual Question Answering (VQA).
Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation
Experiments on KITTI, Cityscapes, and Waymo Open Dataset demonstrate the relevance of our approach and show that our method outperforms state-of-the-art algorithms for the tasks of self-supervised monocular depth estimation, object motion segmentation, monocular scene flow estimation, and visual odometry.
Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation
The scheme first clusters compound target data based on style, discovering multiple latent domains (discover).
Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs
Thus, it is critical for the community to know whether the newly proposed ViT and MLP-Mixer are also vulnerable to adversarial attacks.
Early Stop And Adversarial Training Yield Better surrogate Model: Very Non-Robust Features Harm Adversarial Transferability
The transferability of adversarial examples (AE); known as adversarial transferability, has attracted significant attention because it can be exploited for TransferableBlack-box Attacks (TBA).
How Does SimSiam Avoid Collapse Without Negative Samples? Towards a Unified Understanding of Progress in SSL
Towards avoiding collapse in self-supervised learning (SSL), contrastive loss is widely used but often requires a large number of negative samples.
ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection
A common problem in the task of human-object interaction (HOI) detection is that numerous HOI classes have only a small number of labeled examples, resulting in training sets with a long-tailed distribution.
Ranked #41 on
Human-Object Interaction Detection
on HICO-DET
Category-Level Metric Scale Object Shape and Pose Estimation
Our framework has two branches: the Metric Scale Object Shape branch (MSOS) and the Normalized Object Coordinate Space branch (NOCS).
VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction
To reconstruct a 3D scene from a set of calibrated views, traditional multi-view stereo techniques rely on two distinct stages: local depth maps computation and global depth maps fusion.
Learning Open-World Object Proposals without Learning to Classify
In this paper, we identify that the problem is that the binary classifiers in existing proposal methods tend to overfit to the training categories.
Ranked #2 on
Open World Object Detection
on COCO VOC to non-VOC
LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation
In order to find the uncertain points, we generate an inconsistency mask using the proposed adaptive pixel selector and we label these segment-based regions to achieve near supervised performance with only a small fraction (about 2. 2%) ground truth points, which we call "Segment based Pixel-Labeling (SPL)".
Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation
Volumetric deep learning approach towards stereo matching aggregates a cost volume computed from input left and right images using 3D convolutions.
MCDAL: Maximum Classifier Discrepancy for Active Learning
Recent state-of-the-art active learning methods have mostly leveraged Generative Adversarial Networks (GAN) for sample acquisition; however, GAN is usually known to suffer from instability and sensitivity to hyper-parameters.
Unsupervised Domain Adaptation for Video Semantic Segmentation
In this work, we present a new video extension of this task, namely Unsupervised Domain Adaptation for Video Semantic Segmentation.
Learning to Associate Every Segment for Video Panoptic Segmentation
Temporal correspondence - linking pixels or objects across frames - is a fundamental supervisory signal for the video models.
Ranked #6 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning
The capability of the traditional semi-supervised learning (SSL) methods is far from real-world application due to severely biased pseudo-labels caused by (1) class imbalance and (2) class distribution mismatch between labeled and unlabeled data.
Restoration of Video Frames from a Single Blurred Image with Motion Understanding
We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors, and the blurred image as an observation.
Depth Completion using Plane-Residual Representation
The basic framework of depth completion is to predict a pixel-wise dense depth map using very sparse input data.
Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillation
In this work, we address the issues of missing modalities that have arisen from the Visual Question Answer-Difference prediction task and find a novel method to solve the task at hand.
Universal Adversarial Training with Class-Wise Perturbations
The SOTA universal adversarial training (UAT) method optimizes a single perturbation for all training samples in the mini-batch.
Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation
The key idea of our network is to exploit sparse and accurate point clouds as a cue for guiding correspondences of stereo images in a unified 3D volume space.
Stereo Object Matching Network
This paper presents a stereo object matching method that exploits both 2D contextual information from images as well as 3D object-level information.
Optical Flow Estimation from a Single Motion-blurred Image
A flow estimator network is then used to estimate optical flow from the decoded features in a coarse-to-fine manner.
Motion-blurred Video Interpolation and Extrapolation
Abrupt motion of camera or objects in a scene result in a blurry video, and therefore recovering high quality video requires two types of enhancements: visual enhancement and temporal upsampling.
A Survey On Universal Adversarial Attack
The intriguing phenomenon of adversarial examples has attracted significant attention in machine learning and what might be more surprising to the community is the existence of universal adversarial perturbations (UAPs), i. e. a single perturbation to fool the target DNN for most images.
A Brief Survey on Deep Learning Based Data Hiding
Data hiding is the art of concealing messages with limited perceptual changes.
Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss
Based on the proposed module, the photometric consistency loss can provide complementary self-supervision to networks.
Universal Adversarial Perturbations Through the Lens of Deep Steganography: Towards A Fourier Perspective
We perform task-specific and joint analysis and reveal that (a) frequency is a key factor that influences their performance based on the proposed entropy metric for quantifying the frequency distribution; (b) their success can be attributed to a DNN being highly sensitive to high-frequency content.
Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency
We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision.
Ranked #2 on
Monocular Depth Estimation
on Cityscapes
Learning Representations by Contrasting Clusters While Bootstrapping Instances
Learning visual representations using large-scale unlabelled images is a holy grail for most of computer vision tasks.
Data-Free Universal Adversarial Perturbation and Black-Box Attack
For a more practical universal attack, our investigation of untargeted UAP focuses on alleviating the dependence on the original training samples, from removing the need for sample labels to limiting the sample size.
Towards Robust Data Hiding Against (JPEG) Compression: A Pseudo-Differentiable Deep Learning Approach
Recently, deep learning has shown large success in data hiding, while non-differentiability of JPEG makes it challenging to train a deep pipeline for improving robustness against lossy compression.
An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Search
Deep reinforcement learning (DRL) algorithms and evolution strategies (ES) have been applied to various tasks, showing excellent performances.
The Devil is in the Boundary: Exploiting Boundary Representation for Basis-based Instance Segmentation
In this work, we propose Boundary Basis based Instance Segmentation(B2Inst) to learn a global boundary representation that can complement existing global-mask-based methods that are often lacking high-frequency details.
Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy
Adversarial training is the most widely used technique for improving adversarial robustness to strong white-box attacks.
Dense Relational Image Captioning via Multi-task Triple-Stream Networks
To this end, we propose the multi-task triple-stream network (MTTSNet) which consists of three recurrent units responsible for each POS which is trained by jointly predicting the correct captions and POS for each word.
Double Targeted Universal Adversarial Perturbations
This universal perturbation attacks one targeted source class to sink class, while having a limited adversarial effect on other non-targeted source classes, for avoiding raising suspicions.
CD-UAP: Class Discriminative Universal Adversarial Perturbation
Since the proposed attack generates a universal adversarial perturbation that is discriminative to targeted and non-targeted classes, we term it class discriminative universal adversarial perturbation (CD-UAP).
Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective
This work attempts to understand the impact of BN on DNNs from a non-robust feature perspective.
Revisiting Batch Normalization for Improving Corruption Robustness
We find that simply estimating and adapting the BN statistics on a few (32 for instance) representation samples, without retraining the model, improves the corruption robustness by a large margin on several benchmark datasets with a wide range of model architectures.
Non-Local Spatial Propagation Network for Depth Completion
In this paper, we propose a robust and efficient end-to-end non-local spatial propagation network for depth completion.
Ranked #1 on
Depth Completion
on NYU-Depth V2
Detecting Human-Object Interactions with Action Co-occurrence Priors
A common problem in human-object interaction (HOI) detection task is that numerous HOI classes have only a small number of labeled examples, resulting in training sets with a long-tailed distribution.
Video Panoptic Segmentation
In this paper, we propose and explore a new video extension of this task, called video panoptic segmentation.
Ranked #7 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision
Finally, to decrease the intra-domain gap, we propose to employ a self-supervised adaptation technique from the easy to the hard split.
Ranked #2 on
Domain Adaptation
on Synscapes-to-Cityscapes
Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling
In this paper, we propose to explicitly learn to imagine a storyline that bridges the visual gap.
Ranked #7 on
Visual Storytelling
on VIST
Instance-wise Depth and Motion Learning from Monocular Videos
We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision.
Learning to Localize Sound Sources in Visual Scenes: Analysis and Applications
Visual events are usually accompanied by sounds in our daily lives.
Visuomotor Understanding for Representation Learning of Driving Scenes
We postulate that success on this task requires the network to learn semantic and geometric knowledge in the ego-centric view.
Learning Residual Flow as Dynamic Motion from Stereo Videos
Based on rigid projective geometry, the estimated stereo depth is used to guide the camera motion estimation, and the depth and camera motion are used to guide the residual flow estimation.
Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
To this end, our proposed semi-supervised learning method assigns pseudo-labels to unpaired samples via Generative Adversarial Networks to learn the joint distribution of image and caption.
Deep Iterative Frame Interpolation for Full-frame Video Stabilization
We present a novel deep approach to video stabilization which can generate video frames without cropping and low distortion.
Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video Translation
In this paper, we investigate the problem of unpaired video-to-video translation.
Propose-and-Attend Single Shot Detector
We present a simple yet effective prediction module for a one-stage detector.
Camera Exposure Control for Robust Robot Vision with Noise-Aware Image Quality Assessment
In this paper, we propose a noise-aware exposure control algorithm for robust robot vision.
Align-and-Attend Network for Globally and Locally Coherent Video Inpainting
Our video inpainting network consists of two stages.
Learning Loss for Active Learning
In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks.
Ranked #3 on
Active Learning
on CIFAR10 (10,000)
Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence
Blind video decaptioning is a problem of automatically removing text overlays and inpainting the occluded parts in videos without any input masks.
Deep Video Inpainting
Video inpainting aims to fill spatio-temporal holes with plausible content in a video.
Ranked #7 on
Video Inpainting
on DAVIS
DPSNet: End-to-end Deep Plane Sweep Stereo
The cost volume is constructed using a differentiable warping process that allows for end-to-end training of the network.
Variational Prototyping-Encoder: One-Shot Learning with Prototypical Images
We take an approach to learn a generalizable embedding space for novel tasks.
Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioning
Our goal in this work is to train an image captioning model that generates more dense and informative captions.
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
Self-supervised tasks such as colorization, inpainting and zigsaw puzzle have been utilized for visual representation learning for still images, when the number of labeled images is limited or absent at all.
Ranked #42 on
Self-Supervised Action Recognition
on HMDB51
Discriminative Feature Learning for Unsupervised Video Summarization
The proposed variance loss allows a network to predict output scores for each frame with high discrepancy which enables effective feature learning and significantly improves model performance.
Ranked #3 on
Unsupervised Video Summarization
on SumMe
Supervised Video Summarization
Unsupervised Video Summarization
LinkNet: Relational Embedding for Scene Graph
In this paper, we present a method that improves scene graph generation by explicitly modeling inter-dependency among the entire object instances.
BAM: Bottleneck Attention Module
In this work, we focus on the effect of attention in general deep neural networks.
CBAM: Convolutional Block Attention Module
We propose Convolutional Block Attention Module (CBAM), a simple yet effective attention module for feed-forward convolutional neural networks.
Globally Optimal Inlier Set Maximization for Atlanta Frame Estimation
In this work, we describe man-made structures via an appropriate structure assumption, called Atlanta world, which contains a vertical direction (typically the gravity direction) and a set of horizontal directions orthogonal to the vertical direction.
Efficient adaptive non-maximal suppression algorithms for homogeneous spatial keypoint distribution
Keypoint detection usually results in a large number of keypoints which are mostly clustered, redundant, and noisy.
Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning
In addition, we present a 'distort-and-recover' training scheme which only requires high-quality reference images for training instead of input and retouched image pairs.
EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images
Light field cameras capture both the spatial and the angular properties of light rays in space.
Robust Depth Estimation from Auto Bracketed Images
As demand for advanced photographic applications on hand-held devices grows, these electronics require the capture of high quality depth.
Learning to Localize Sound Source in Visual Scenes
We show that even with a few supervision, false conclusion is able to be corrected and the source of sound in a visual scene can be localized effectively.
Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks
Human behavior understanding is arguably one of the most important mid-level components in artificial intelligence.
Learning Image Representations by Completing Damaged Jigsaw Puzzles
The recovery of the aforementioned damage pushes the network to obtain robust and general-purpose representations.
Co-domain Embedding using Deep Quadruplet Networks for Unseen Traffic Sign Recognition
Recent advances in visual recognition show overarching success by virtue of large amounts of supervised data.
Intelligent Assistant for People with Low Vision Abilities
This paper proposes a wearable system for visually impaired people that can be utilized to obtain an extensive feedback about their surrounding environment.
Light-weight place recognition and loop detection using road markings
In this paper, we propose an efficient algorithm for robust place recognition and loop detection using camera information only.
VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition
In this paper, we propose a unified end-to-end trainable multi-task network that jointly handles lane and road marking detection and recognition that is guided by a vanishing point under adverse weather conditions.
Ranked #1 on
Lane Detection
on Caltech Lanes Washington
Deltille Grids for Geometric Camera Calibration
The recent proliferation of high resolution cameras presents an opportunity to achieve unprecedented levels of precision in visual 3D reconstruction.
StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection
One-stage object detectors such as SSD or YOLO already have shown promising accuracy with small memory footprint and fast speed.
Gradient-based Camera Exposure Control for Outdoor Mobile Platforms
We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms.
Pixel-Level Matching for Video Object Segmentation using Convolutional Neural Networks
We propose a novel video object segmentation algorithm based on pixel-level matching using Convolutional Neural Networks (CNN).
Ranked #73 on
Semi-Supervised Video Object Segmentation
on DAVIS 2016
Weakly- and Self-Supervised Learning for Content-Aware Deep Image Retargeting
Our method implicitly learns an attention map, which leads to a content-aware shift map for image retargeting.
Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning
Cinemagraphs are a compelling way to convey dynamic aspects of a scene.
Two-Phase Learning for Weakly Supervised Object Localization
Weakly supervised semantic segmentation and localiza- tion have a problem of focusing only on the most important parts of an image since they use only image-level annota- tions.
Noise Robust Depth From Focus Using a Ring Difference Filter
With the result from the FM, the role of a DfF pipeline is to determine and recalculate unreliable measurements while enhancing those that are reliable.
A Unified Approach of Multi-scale Deep and Hand-crafted Features for Defocus Estimation
In this paper, we introduce robust and synergetic hand-crafted features and a simple but efficient deep feature from a convolutional neural network (CNN) architecture for defocus estimation.
Ranked #2 on
Defocus Estimation
on CUHK - Blur Detection Dataset
Contextually Customized Video Summaries via Natural Language
Despite the challenging baselines, our method still manages to show comparable or even exceeding performance.
Action-Driven Object Detection with Top-Down Visual Attentions
In this paper, we present an "action-driven" detection mechanism using our "top-down" visual attention model.
Refining Geometry from Depth Sensors using IR Shading Images
To resolve the ambiguity in our model between the normals and distances, we utilize an initial 3D mesh from the Kinect fusion and multi-view information to reliably estimate surface details that were not captured and reconstructed by the Kinect fusion.
3D Display Calibration by Visual Pattern Analysis
Experimental results demonstrate that our method is quite accurate, about a half order of magnitude higher than prior work; is efficient, spending less than 2 s for computation; and is robust to noise, working well in the SNR regime as low as 6 dB.
Simultaneous Estimation of Near IR BRDF and Fine-Scale Surface Geometry
Near-Infrared (NIR) images of most materials exhibit less texture or albedo variations making them beneficial for vision tasks such as intrinsic image decomposition and structured light depth estimation.
Globally Optimal Manhattan Frame Estimation in Real-Time
Given a set of surface normals, we pose a Manhattan Frame (MF) estimation problem as a consensus set maximization that maximizes the number of inliers over the rotation search space.
Efficient and Robust Color Consistency for Community Photo Collections
We present a robust low-rank matrix factorization method to estimate the unknown parameters of this model.
Stereo Matching With Color and Monochrome Cameras in Low-Light Conditions
Consumer devices with stereo cameras have become popular because of their low-cost depth sensing capability.
Video-Story Composition via Plot Analysis
Inspired by plot analysis of written stories, our method generates a sequence of video clips ordered in such a way that it reflects plot dynamics and content coherency.
High-Quality Depth From Uncalibrated Small Motion Clip
We propose a novel approach that generates a high-quality depth map from a set of images captured with a small viewpoint variation, namely small motion clip.
Robust and Globally Optimal Manhattan Frame Estimation in Near Real Time
Most man-made environments, such as urban and indoor scenes, consist of a set of parallel and orthogonal planar structures.
Fine-scale Surface Normal Estimation using a Single NIR Image
We present surface normal estimation using a single near infrared (NIR) image.
Human Attention Estimation for Natural Images: An Automatic Gaze Refinement Approach
Photo collections and its applications today attempt to reflect user interactions in various forms.
Pseudo-Bayesian Robust PCA: Algorithms and Analyses
Commonly used in computer vision and other applications, robust PCA represents an algorithmic attempt to reduce the sensitivity of classical PCA to outliers.
Accurate Camera Calibration Robust to Defocus Using a Smartphone
We propose a novel camera calibration method for defocused images using a smartphone under the assumption that the defocus blur is modeled as a convolution of a sharp image with a Gaussian point spread function (PSF).
Complementary Sets of Shutter Sequences for Motion Deblurring
In this paper, we present a novel multi-image motion deblurring method utilizing the coded exposure technique.
High Quality Structure From Small Motion for Rolling Shutter Cameras
To address these problems, we introduce a novel 3D reconstruction method from narrow-baseline image sequences that effectively handles the effects of a rolling shutter that occur from most of commercial digital cameras.
Automatic Content-Aware Color and Tone Stylization
We introduce a new technique that automatically generates diverse, visually compelling stylizations for a photograph in an unsupervised manner.
Vision System and Depth Processing for DRC-HUBO+
This paper presents a vision system and a depth processing algorithm for DRC-HUBO+, the winner of the DRC finals 2015.
Fast Randomized Singular Value Thresholding for Low-rank Optimization
The problems related to NNM, or WNNM, can be solved iteratively by applying a closed-form proximal operator, called Singular Value Thresholding (SVT), or Weighted SVT, but they suffer from high computational cost of Singular Value Decomposition (SVD) at each iteration.
AttentionNet: Aggregating Weak Directions for Accurate Object Detection
We present a novel detection method using a deep convolutional neural network (CNN), named AttentionNet.
One-Day Outdoor Photometric Stereo via Skylight Estimation
We present an outdoor photometric stereo method using images captured in a single day.
Accurate Depth Map Estimation From a Lenslet Light Field Camera
This paper introduces an algorithm that accurately estimates depth maps using a lenslet light field camera.
Multispectral Pedestrian Detection: Benchmark Dataset and Baseline
With this dataset, we introduce multispectral ACF, which is an extension of aggregated channel features (ACF) to simultaneously handle color-thermal image pairs.
Fast Randomized Singular Value Thresholding for Nuclear Norm Minimization
The problems related to NNM (or WNNM) can be solved iteratively by applying a closed-form proximal operator, called Singular Value Thresholding (SVT) (or Weighted SVT), but they suffer from high computational cost to compute a Singular Value Decomposition (SVD) at each iteration.
Partial Sum Minimization of Singular Values in Robust PCA: Algorithm and Applications
Robust Principal Component Analysis (RPCA) via rank minimization is a powerful tool for recovering underlying low-rank structure of clean data corrupted with sparse noise/outliers.
Fisher Kernel for Deep Neural Activations
In this paper, we present a straightforward framework for better image representation by combining the two approaches.
Exploiting Shading Cues in Kinect IR Images for Geometry Refinement
To resolve ambiguity in our model between normals and distance, we utilize an initial 3D mesh from the Kinect fusion and multi-view information to reliably estimate surface details that were not reconstructed by the Kinect fusion.
Color Transfer Using Probabilistic Moving Least Squares
This paper introduces a new color transfer method which is a process of transferring color of an image to match the color of another image of the same scene.
Calibrating a Non-isotropic Near Point Light Source using a Plane
We show that a non-isotropic near point light source rigidly attached to a camera can be calibrated using multiple images of a weakly textured planar scene.
