Search Results for author:
Found 61 papers, 26 papers with code
ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipation
In this work, we tackle these two problems, action segmentation and action anticipation, jointly using a unified diffusion model dubbed ActFusion.
Bootstrapping Top-down Information for Self-modulating Slot Attention
Object-centric learning (OCL) aims to learn representations of individual objects within visual scenes without manual supervision, facilitating efficient and effective visual reasoning.
PLOT: Text-based Person Search with Part Slot Attention for Corresponding Part Discovery
Text-based person search, employing free-form text queries to identify individuals within a vast image collection, presents a unique challenge in aligning visual and textual representations, particularly at the human part level.
Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization
We study the problem of training an unbiased and accurate model given a dataset with multiple biases.
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models
Large-scale image-text pre-trained models enable zero-shot classification and provide consistent accuracy across various data distributions.
Online Temporal Action Localization with Memory-Augmented Transformer
We also propose a novel action localization method that observes the current input segment to predict the end time of the ongoing action and accesses the memory queue to estimate the start time of the action.
Classification Matters: Improving Video Action Detection with Class-Specific Attention
Video action detection (VAD) aims to detect actors and classify their actions in a video.
FREST: Feature RESToration for Semantic Segmentation under Multiple Adverse Conditions
To address this challenging task in practical scenarios where labeled normal condition images are not accessible in training, we propose FREST, a novel feature restoration framework for source-free domain adaptation (SFDA) of semantic segmentation to adverse conditions.
Extreme Point Supervised Instance Segmentation
Our work considers extreme points as a part of the true instance mask and propagates them to identify potential foreground and background points, which are all together used for training a pseudo label generator.
Distilling Diffusion Models into Conditional GANs
We propose a method to distill a complex multistep diffusion model into a single-step conditional GAN student model, dramatically accelerating inference, while preserving image quality.
Active Label Correction for Semantic Segmentation with Foundation Models
Training and validating models for semantic segmentation require datasets with pixel-wise annotations, which are notoriously labor-intensive.
Activity Grammars for Temporal Action Segmentation
Sequence prediction on temporal data requires the ability to understand compositional structures of multi-level semantics beyond individual and contextual properties.
Towards More Practical Group Activity Detection: A New Benchmark and Model
Group activity detection (GAD) is the task of identifying members of each group and classifying the activity of the group at the same time in a video.
Universal Metric Learning with Parameter-Efficient Transfer Learning
In this regard, we introduce a novel metric learning paradigm, called Universal Metric Learning (UML), which learns a unified distance metric capable of capturing relations across multiple data distributions.
Shatter and Gather: Learning Referring Image Segmentation with Text Supervision
Referring image segmentation, the task of segmenting any arbitrary entities described in free-form texts, opens up a variety of vision applications.
SYNAuG: Exploiting Synthetic Data for Data Imbalance Problems
Data imbalance in training data often leads to biased predictions from trained models, which in turn causes ethical and social issues.
PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization
In a joint vision-language space, a text feature (e. g., from "a photo of a dog") could effectively represent its relevant image features (e. g., from dog photos).
Ranked #1 on
Domain Generalization
on DomainNet
Extending CLIP's Image-Text Alignment to Referring Image Segmentation
Referring Image Segmentation (RIS) is a cross-modal task that aims to segment an instance described by a natural language expression.
Adaptive Superpixel for Active Learning in Semantic Segmentation
Learning semantic segmentation requires pixel-wise annotations, which can be time-consuming and expensive.
Human Pose Estimation in Extremely Low-Light Conditions
We study human pose estimation in extremely low-light images.
HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization
Supervision for metric learning has long been given in the form of equivalence between human-labeled classes.
Learning to Detect Semantic Boundaries with Image-level Class Labels
This paper presents the first attempt to learn semantic boundary detection using image-level class labels as supervision.
Improving Cross-Modal Retrieval with Set of Diverse Embeddings
It seeks to encode a sample into a set of different embedding vectors that capture different semantics of the sample.
Cross-Domain Ensemble Distillation for Domain Generalization
Domain generalization is the task of learning models that generalize to unseen target domains.
Ranked #5 on
Image to sketch recognition
on PACS
Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval
Metric learning aims to build a distance metric typically by learning an effective embedding function that maps similar objects into nearby points in its embedding space.
Combating Label Distribution Shift for Active Domain Adaptation
We consider the problem of active domain adaptation (ADA) to unlabeled target data, of which subset is actively selected and labeled given a budget constraint.
Learning Debiased Classifier with Biased Committee
We propose a new method for training debiased classifiers with no spurious attribute label.
Self-Taught Metric Learning without Labels
At the heart of our framework lies an algorithm that investigates contexts of data on the embedding space to predict their class-equivalence relations as pseudo labels.
Detector-Free Weakly Supervised Group Activity Recognition
Group activity recognition is the task of understanding the activity conducted by a group of people as a whole in a multi-person video.
Semi-supervised Semantic Segmentation with Error Localization Network
This paper studies semi-supervised learning of semantic segmentation, which assumes that only a small portion of training images are labeled and the others remain unlabeled.
FIFO: Learning Fog-invariant Features for Foggy Scene Segmentation
Robust visual recognition under adverse weather conditions is of great importance in real-world applications.
Ranked #4 on
Domain Adaptation
on Cityscapes-to-FoggyDriving
Reflection and Rotation Symmetry Detection via Equivariant Learning
The inherent challenge of detecting symmetries stems from arbitrary orientations of symmetry patterns; a reflection symmetry mirrors itself against an axis with a specific orientation while a rotation symmetry matches its rotated copy with a specific orientation.
ReSTR: Convolution-free Referring Image Segmentation Using Transformers
Most of existing methods for this task rely heavily on convolutional neural networks, which however have trouble capturing long-range dependencies between entities in the language expression and are not flexible enough for modeling interactions between the two different modalities.
Collaborative Transformers for Grounded Situation Recognition
To implement this idea, we propose Collaborative Glance-Gaze TransFormer (CoFormer) that consists of two modules: Glance transformer for activity classification and Gaze transformer for entity estimation.
Ranked #2 on
Situation Recognition
on imSitu
Learning to Generate Novel Classes for Deep Metric Learning
Motivated by this, we introduce a new data augmentation approach that synthesizes novel classes and their embedding vectors.
Style Neophile: Constantly Seeking Novel Styles for Domain Generalization
Existing methods in this direction suppose that a domain can be characterized by styles of its images, and train a network using style-augmented data so that the network is not biased to particular style distributions.
Grounded Situation Recognition with Transformers
Grounded Situation Recognition (GSR) is the task that not only classifies a salient action (verb), but also predicts entities (nouns) associated with semantic roles and their locations in the given image.
Ranked #5 on
Situation Recognition
on imSitu
Relational Self-Attention: What's Missing in Attention for Video Understanding
Convolution has been arguably the most important feature transform for modern neural networks, leading to the advance of deep learning.
Ranked #12 on
Action Recognition
on Diving-48
Cross Domain Ensemble Distillation for Domain Generalization
For domain generalization, the task of learning a model that generalizes to unseen target domains utilizing multiple source domains, many approaches explicitly align the distribution of the domains.
Ranked #38 on
Domain Generalization
on Office-Home
WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
We also present a method which injects styles of the web-crawled images into training images on-the-fly during training, which enables the network to experience images of diverse styles with reliable labels for effective training.
ASMR: Learning Attribute-Based Person Search with Adaptive Semantic Margin Regularizer
Attribute-based person search is the task of finding person images that are best matched with a set of text attributes given as query.
On The Distribution of Penultimate Activations of Classification Networks
This paper studies probability distributions of penultimate activations of classification networks.
Embedding Transfer with Label Relaxation for Improved Metric Learning
Our method exploits pairwise similarities between samples in the source embedding space as the knowledge, and transfers them through a loss used for learning target embedding models.
Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognition
With a sufficient volume of the neighborhood in space and time, it effectively captures long-term interaction and fast motion in the video, leading to robust action recognition.
Ranked #19 on
Action Recognition
on Something-Something V1
(using extra training data)
Learning Self-Similarity in Space and Time as a Generalized Motion for Action Recognition
We leverage the whole volume of STSS and let our model learn to extract an effective motion representation from it.
Embedding Transfer via Smooth Contrastive Loss
To this end, we design a new loss called smooth contrastive loss, which pulls together or pushes apart a pair of samples in a target embedding space with strength determined by their semantic similarity in the source embedding space; an analysis of the loss reveals that this property enables more important pairs to contribute more to learning the target embedding space.
MotionSqueeze: Neural Motion Feature Learning for Video Understanding
As the frame-by-frame optical flows require heavy computation, incorporating motion information has remained a major computational bottleneck for video understanding.
Ranked #1 on
Video Classification
on Something-Something V2
URIE: Universal Image Enhancement for Visual Recognition in the Wild
Despite the great advances in visual recognition, it has been witnessed that recognition models trained on clean images of common datasets are not robust against distorted images in the real world.
Proxy Anchor Loss for Deep Metric Learning
The former class can leverage fine-grained semantic relations between data points, but slows convergence in general due to its high training complexity.
Ranked #10 on
Metric Learning
on CUB-200-2011
(using extra training data)
Fine-Grained Image Classification
Fine-Grained Vehicle Classification
+1
Domain-Specific Batch Normalization for Unsupervised Domain Adaptation
In the first stage, we estimate pseudo-labels for the examples in the target domain using an external unsupervised domain adaptation algorithm---for example, MSTN or CPUA---integrating the proposed domain-specific batch normalization.
Deep Metric Learning Beyond Binary Supervision
Metric Learning for visual similarity has mostly adopted binary supervision indicating whether a pair of images are of the same class or not.
Universal Bounding Box Regression and Its Applications
Bounding-box regression is a popular technique to refine or predict localization boxes in recent object detection approaches.
Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations
For generating the pseudo labels, we first identify confident seed areas of object classes from attention maps of an image classification model, and propagate them to discover the entire instance areas with accurate boundaries.
Image Classification
Image-level Supervised Instance Segmentation
+2
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation
To alleviate this issue, we present a novel framework that generates segmentation labels of images given their image-level class labels.
Weakly Supervised Semantic Segmentation using Web-Crawled Videos
Our goal is to overcome this limitation with no additional human intervention by retrieving videos relevant to target class labels from web repository, and generating segmentation labels from the retrieved videos to simulate strong supervision for semantic segmentation.
Thin-Slicing for Pose: Learning to Understand Pose Without Explicit Pose Estimation
We address the problem of learning a pose-aware, compact embedding that projects images with similar human poses to be placed close-by in the embedding space.
Unsupervised Object Discovery and Tracking in Video Collections
This paper addresses the problem of automatically localizing dominant objects as spatio-temporal tubes in a noisy collection of videos with minimal or even no supervision.
Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network
We propose an online visual tracking algorithm by learning discriminative saliency map using Convolutional Neural Network (CNN).
Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals
This paper addresses unsupervised discovery and localization of dominant objects from a noisy image collection with multiple object classes.
Object Localization based on Structural SVM using Privileged Information
We propose a structured prediction algorithm for object localization based on Support Vector Machines (SVMs) using privileged information.
Multi-agent Event Detection: Localization and Role Assignment
We present a joint estimation technique of event localization and role assignment when the target video event is described by a scenario.
