Search Results for author:
Found 89 papers, 36 papers with code
URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark
We propose a unified referring video object segmentation network (URVOS).
Ranked #7 on
Referring Video Object Segmentation
on MeViS
Traffic Accident Benchmark for Causality Recognition
We propose a brand new benchmark for analyzing causality in traffic accident videos by decomposing an accident into a pair of events, cause and effect.
Task-Aware Quantization Network for JPEG Image Compression
We propose to learn a deep neural network for JPEG image compression, which predicts image-specific optimized quantization tables fully compatible with the standard JPEG encoder and decoder.
Diffusion-Based Image-to-Image Translation by Noise Correction via Prompt Interpolation
Our approach revises the original noise prediction network of a pretrained diffusion model by introducing a noise correction term.
Revisiting Machine Unlearning with Dimensional Alignment
Based on these findings, we introduce a novel evaluation metric for machine unlearning, coined dimensional alignment, which measures the alignment between the eigenspaces of the forget and retain set samples.
FIFO-Diffusion: Generating Infinite Videos from Text without Training
We propose a novel inference technique based on a pretrained diffusion model for text-conditional video generation.
Leveraging Temporal Contextualization for Video Action Recognition
To be specific, we introduce Temporal Contextualization (TC), a layer-wise temporal information infusion mechanism for videos, which 1) extracts core information from each frame, 2) connects relevant information across frames for the summarization into context tokens, and 3) leverages the context tokens for feature encoding.
Ranked #1 on
Zero-Shot Action Recognition
on Kinetics
A Training-Free Defense Framework for Robust Learned Image Compression
We study the robustness of learned image compression models against adversarial attacks and present a training-free defense technique based on simple image transform functions.
Relaxed Contrastive Learning for Federated Learning
We propose a novel contrastive learning framework to effectively address the challenges of data heterogeneity in federated learning.
Learning with Noisy Labels: Interconnection of Two Expectation-Maximizations
In this framework, we provide a proper objective function and an optimization algorithm based on two expectation-maximization (EM) cycles.
Robust Image Denoising through Adversarial Frequency Mixup
AFM leverages mixup in the frequency domain to generate noisy images with distinctive and challenging noise characteristics all the while preserving the properties of authentic real-world noise.
Observation-Guided Diffusion Probabilistic Models
We propose a novel diffusion-based image generation method called the observation-guided diffusion probabilistic model (OGDM), which effectively addresses the tradeoff between quality control and fast sampling.
NICE: CVPR 2023 Challenge on Zero-shot Image Captioning
In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge.
ContraNeRF: 3D-Aware Generative Model via Contrastive Learning with Unsupervised Implicit Pose Embedding
Although 3D-aware GANs based on neural radiance fields have achieved competitive performance, their applicability is still limited to objects or scenes with the ground-truths or prediction models for clearly defined canonical camera poses.
On the Stability-Plasticity Dilemma of Class-Incremental Learning
A primary goal of class-incremental learning is to strike a balance between stability and plasticity, where models should be both stable enough to retain knowledge learned from previously seen classes, and plastic enough to learn concepts from new classes.
Randomized Adversarial Style Perturbations for Domain Generalization
We propose a novel domain generalization technique, referred to as Randomized Adversarial Style Perturbation (RASP), which is motivated by the observation that the characteristics of each domain are captured by the feature statistics corresponding to style.
Cross-Class Feature Augmentation for Class Incremental Learning
The proposed approach has a unique perspective to utilize the previous knowledge in class incremental learning since it augments features of arbitrary target classes using examples in other classes via adversarial attacks on a previously learned classifier.
Multi-Modal Representation Learning with Text-Driven Soft Masks
We propose a visual-linguistic representation learning approach within a self-supervised learning framework by introducing a new operation, loss, and data augmentation strategy.
Variational Distribution Learning for Unsupervised Text-to-Image Generation
We propose a text-to-image generation algorithm based on deep neural networks when text captions for images are unavailable during training.
Information-Theoretic GAN Compression with Variational Energy-based Model
We propose an information-theoretic knowledge distillation approach for the compression of generative adversarial networks, which aims to maximize the mutual information between teacher and student networks via a variational optimization based on an energy-based model.
Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense
Experimental results demonstrate that, in terms of adversarial robustness, NIM is superior to MIM thanks to its effective denoising capability.
Online Backfilling with No Regret for Large-Scale Image Retrieval
Backfilling is the process of re-extracting all gallery embeddings from upgraded models in image retrieval systems.
Towards Sequence-Level Training for Visual Tracking
Despite the extensive adoption of machine learning on the task of visual object tracking, recent learning-based approaches have largely overlooked the fact that visual tracking is a sequence-level task in its nature; they rely heavily on frame-level training, which inevitably induces inconsistency between training and testing in terms of both data distributions and task objectives.
Ranked #19 on
Visual Object Tracking
on TrackingNet
Multi-Level Branched Regularization for Federated Learning
A critical challenge of federated learning is data heterogeneity and imbalance across clients, which leads to inconsistency between local networks and unstable convergence of global models.
Pooling Revisited: Your Receptive Field is Suboptimal
The size and shape of the receptive field determine how the network aggregates local information and affect the overall performance of a model considerably.
Online Hybrid Lightweight Representations Learning: Its Application to Visual Tracking
This paper presents a novel hybrid representation learning framework for streaming data, where an image frame in a video is modeled by an ensemble of two distinct deep neural networks; one is a low-bit quantized network and the other is a lightweight full-precision network.
Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation
We present a novel class incremental learning approach based on deep neural networks, which continually learns new tasks with limited memory for storing examples in the previous tasks.
Class-Incremental Learning for Action Recognition in Videos
We tackle catastrophic forgetting problem in the context of class-incremental learning for video recognition, which has not been explored actively despite the popularity of continual learning.
Learning to Adapt to Unseen Abnormal Activities under Weak Supervision
We present a meta-learning framework for weakly supervised anomaly detection in videos, where the detector learns to adapt to unseen types of abnormal activities effectively when only video-level annotations of binary labels are available.
Learning Semantic Segmentation from Multiple Datasets with Label Shifts
First, a gradient conflict in training due to mismatched label spaces is identified and a class-independent binary cross-entropy loss is proposed to alleviate such label conflicts.
Communication-Efficient Federated Learning with Accelerated Client Gradient
To address this challenge, we propose a simple but effective federated learning framework, which improves the consistency across clients and facilitates the convergence of the server model.
Information-Theoretic Bias Reduction via Causal View of Spurious Correlation
We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation, which is effective to identify the feature-level algorithmic bias by taking advantage of conditional mutual information.
Self-Supervised Dense Consistency Regularization for Image-to-Image Translation
Unsupervised image-to-image translation has gained considerable attention due to the recent impressive progress based on generative adversarial networks (GANs).
InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering
We present an information-theoretic regularization technique for few-shot novel view synthesis based on neural implicit representation.
Learning Debiased and Disentangled Representations for Semantic Segmentation
By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes, and the model is able to learn more debiased and disentangled feature representations.
Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform
In addition, the proposed framework allows us to perform task-aware image compressions for various tasks, e. g., classification, by efficiently estimating optimized quality maps specific to target tasks for our encoding network.
Unsupervised Learning of Debiased Representations with Pseudo-Attributes
Dataset bias is a critical challenge in machine learning since it often leads to a negative impact on a model due to the unintended decision rules captured by spurious correlations.
MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators
We propose a framework of generative adversarial networks with multiple discriminators, which collaborate to represent a real dataset more effectively.
Open-Set Representation Learning through Combinatorial Embedding
Visual recognition tasks are often limited to dealing with a small subset of classes simply because the labels for the remaining classes are unavailable.
RaScaNet: Learning Tiny Models by Raster-Scanning Images
RaScaNet reads only a few rows of pixels at a time using a convolutional neural network and then sequentially learns the representation of the whole image using a recurrent neural network.
CoSMo: Content-Style Modulation for Image Retrieval With Text Feedback
We focus on designing an image-text compositor, i. e., integrating multi-modal inputs to produce a representation similar to that of the target image.
Ranked #17 on
Image Retrieval
on Fashion IQ
Exemplar-Based Open-Set Panoptic Segmentation Network
We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task.
Learning Student-Friendly Teacher Networks for Knowledge Distillation
In other words, at the time of optimizing a teacher model, the proposed algorithm learns the student branches jointly to obtain student-friendly representations.
Rotation-Invariant Local-to-Global Representation Learning for 3D Point Cloud
We propose a local-to-global representation learning algorithm for 3D point cloud data, which is appropriate to handle various geometric transformations, especially rotation, without explicit data augmentation with respect to the transformations.
Operation-Aware Soft Channel Pruning using Differentiable Masks
We propose a simple but effective data-driven channel pruning algorithm, which compresses deep neural networks in a differentiable way by exploiting the characteristics of operations.
Local-Global Video-Text Interactions for Temporal Grounding
This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query.
Channel Attention Is All You Need for Video Frame Interpolation
Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion.
Weakly Supervised Instance Segmentation by Deep Community Learning
We present a weakly supervised instance segmentation algorithm based on deep community learning with multiple tasks.
Image-level Supervised Instance Segmentation
object-detection
+4
Combinatorial Inference against Label Noise
Label noise is one of the critical sources that degrade generalization performance of deep neural networks significantly.
Towards Oracle Knowledge Distillation with Neural Architecture Search
We present a novel framework of knowledge distillation that is capable of learning powerful and efficient student models from ensemble teacher networks.
Real-Time Object Tracking via Meta-Learning: Efficient Model Adaptation and One-Shot Channel Pruning
We propose a novel meta-learning framework for real-time object tracking with efficient model adaptation and channel pruning.
Reinforcing an Image Caption Generator Using Off-Line Human Feedback
Human ratings are currently the most accurate way to assess the quality of an image captioning model, yet most often the only used outcome of an expensive human rating evaluation is a few overall statistics over the evaluation dataset.
Fine-Grained Neural Architecture Search
We present an elegant framework of fine-grained neural architecture search (FGNAS), which allows to employ multiple heterogeneous operations within a single layer and can even generate compositional feature maps using several different base operations.
Efficient Decoupled Neural Architecture Search by Structure and Operation Sampling
We propose a novel neural architecture search algorithm via reinforcement learning by decoupling structure and operation search processes.
Regularizing Neural Networks via Stochastic Branch Layers
We introduce a novel stochastic regularization technique for deep neural networks, which decomposes a layer into multiple branches with different parameters and merges stochastically sampled combinations of the outputs from the branches during training.
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Catastrophic forgetting is a critical challenge in training deep neural networks.
Learning to Optimize Domain Specific Normalization for Domain Generalization
We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers that are specific to individual domains.
Ranked #3 on
Unsupervised Domain Adaptation
on PACS
Domain-Specific Batch Normalization for Unsupervised Domain Adaptation
In the first stage, we estimate pseudo-labels for the examples in the target domain using an external unsupervised domain adaptation algorithm---for example, MSTN or CPUA---integrating the proposed domain-specific batch normalization.
Context-Aware Zero-Shot Recognition
We present a novel problem setting in zero-shot learning, zero-shot object recognition and detection in the context.
Streamlined Dense Video Captioning
Dense video captioning is an extremely challenging task since accurate and coherent description of events in a video requires holistic understanding of video contents as well as contextual reasoning of individual events.
Learning to Specialize with Knowledge Distillation for Visual Question Answering
The proposed framework is model-agnostic and applicable to any tasks other than VQA, e. g., image classification with a large number of labels but few per-class examples, which is known to be difficult under existing MCL schemes.
Transfer Learning via Unsupervised Task Discovery for Visual Question Answering
Specifically, we employ linguistic knowledge sources such as structured lexical database (e. g. WordNet) and visual descriptions for unsupervised task discovery, and transfer a learned task conditional visual classifier as an answering unit in a visual question answering model.
Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences
The proposed loss function enables us to learn deep neural networks that predict confidence calibrated scores using a single inference.
Real-Time MDNet
We present a fast and accurate visual tracking algorithm based on the multi-domain convolutional neural network (MDNet).
Attentive Semantic Alignment with Offset-Aware Correlation Kernels
Semantic correspondence is the problem of establishing correspondences across images depicting different instances of the same object or scene class.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
Image geolocalization is the task of identifying the location depicted in a photo based only on its visual information.
Ranked #1 on
Photo geolocation estimation
on Im2GPS
(Reference images metric)
Weakly Supervised Action Localization by Sparse Temporal Pooling Network
We propose a weakly supervised temporal action localization algorithm on untrimmed videos using convolutional neural networks.
Ranked #13 on
Weakly Supervised Action Localization
on ActivityNet-1.3
(mAP@0.5 metric)
Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
Overfitting is one of the most critical challenges in deep neural networks, and there are various types of regularization methods to improve generalization performance.
Visual Reference Resolution using Attention Memory for Visual Dialog
From this memory, the model retrieves the previous attention, taking into account recency, which is most relevant for the current question, in order to resolve potentially ambiguous references.
Ranked #13 on
Visual Dialog
on VisDial v0.9 val
(R@1 metric)
BranchOut: Regularization for Online Ensemble Tracking With Convolutional Neural Networks
We propose an extremely simple but effective regularization technique of convolutional neural networks (CNNs), referred to as BranchOut, for online ensemble tracking.
Multi-Object Tracking With Quadruplet Convolutional Neural Networks
We propose Quadruplet Convolutional Neural Networks (Quad-CNN) for multi-object tracking, which learn to associate object detections across frames using quadruplet losses.
Superpixel-Based Tracking-By-Segmentation Using Markov Chains
We propose a simple but effective tracking-by-segmentation algorithm using Absorbing Markov Chain (AMC) on superpixel segmentation, where target state is estimated by a combination of bottom-up and top-down approaches, and target segmentation is propagated to subsequent frames in a recursive manner.
Weakly Supervised Semantic Segmentation using Web-Crawled Videos
Our goal is to overcome this limitation with no additional human intervention by retrieving videos relevant to target class labels from web repository, and generating segmentation labels from the retrieved videos to simulate strong supervision for semantic segmentation.
Large-Scale Image Retrieval with Attentive Deep Local Features
We propose an attentive local feature descriptor suitable for large-scale image retrieval, referred to as DELF (DEep Local Feature).
Ranked #2 on
Image Retrieval
on Oxf105k
Text-guided Attention Model for Image Captioning
Visual attention plays an important role to understand images and demonstrates its effectiveness in generating natural language descriptions of images.
MarioQA: Answering Questions by Watching Gameplay Videos
To address this objective, we automatically generate a customized synthetic VideoQA dataset using {\em Super Mario Bros.} gameplay videos so that it contains events with different levels of reasoning complexity.
Modeling and Propagating CNNs in a Tree Structure for Visual Tracking
We present an online visual tracking algorithm by managing multiple target appearance models in a tree structure.
Training Recurrent Answering Units with Joint Loss Minimization for VQA
We propose a novel algorithm for visual question answering based on a recurrent deep neural network, where every module in the network corresponds to a complete answering unit with attention mechanism by itself.
Ranked #2 on
Visual Question Answering (VQA)
on VQA v1 test-std
Progressive Attention Networks for Visual Attribute Prediction
We propose a novel attention model that can accurately attends to target objects of various scales and shapes in images.
Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network
We propose a novel weakly-supervised semantic segmentation algorithm based on Deep Convolutional Neural Network (DCNN).
Tracking-by-Segmentation With Online Gradient Boosting Decision Tree
We evaluate the performance of our tracking algorithm based on the measures for segmentation masks, where our algorithm illustrates superior accuracy compared to the state-of-the-art segmentation-based tracking methods.
Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction
We tackle image question answering (ImageQA) problem by learning a convolutional neural network (CNN) with a dynamic parameter layer whose weights are determined adaptively based on questions.
Image Retrieval with Multi-Modal Query
Parameter Prediction
+2
Learning Multi-Domain Convolutional Neural Networks for Visual Tracking
Our algorithm pretrains a CNN using a large set of videos with tracking ground-truths to obtain a generic target representation.
Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
We propose a novel deep neural network architecture for semi-supervised semantic segmentation using heterogeneous annotations.
Learning to Select Pre-Trained Deep Representations with Bayesian Evidence Framework
We propose a Bayesian evidence framework to facilitate transfer learning from pre-trained deep convolutional neural networks (CNNs).
Learning Deconvolution Network for Semantic Segmentation
We propose a novel semantic segmentation algorithm by learning a deconvolution network.
Ranked #3 on
Curved Text Detection
on SCUT-CTW1500
Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network
We propose an online visual tracking algorithm by learning discriminative saliency map using Convolutional Neural Network (CNN).
Object Localization based on Structural SVM using Privileged Information
We propose a structured prediction algorithm for object localization based on Support Vector Machines (SVMs) using privileged information.
Multi-agent Event Detection: Localization and Role Assignment
We present a joint estimation technique of event localization and role assignment when the target video event is described by a scenario.
