Search Results for author:
Found 91 papers, 40 papers with code
Polar Prototype Networks
The structure is defined by polar prototypes, points on the hypersphere of the output space.
Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Models
Most existing methods approach it as a multi-class classification problem, which restricts the outcome to a predefined closed-set of curated answers.
Energy-Based Test Sample Adaptation for Domain Generalization
In this paper, we propose energy-based sample adaptation at test time for domain generalization.
Unlocking Slot Attention by Changing Optimal Transport Costs
Slot attention is a powerful method for object-centric modeling in images and videos.
Test of Time: Instilling Video-Language Models with a Sense of Time
Our work serves as a first step towards probing and instilling a sense of time in existing video-language models without the need for data and compute-intense training from scratch.
Detecting Objects with Graph Priors and Graph Refinement
The key idea of our paper is to model object relations as a function of initial class predictions and co-occurrence priors to generate a graph representation of an image for improved classification and bounding box regression.
Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight
Our goal is to recognize activities in the dark as well as in the day.
Variational Model Perturbation for Source-Free Domain Adaptation
Our model perturbation provides a new probabilistic way for domain adaptation which enables efficient adaptation to target domains while maximally preserving knowledge in source models.
Self-Guided Diffusion Models
Diffusion models have demonstrated remarkable progress in image generation quality, especially when guidance is used to control the generative process.
Association Graph Learning for Multi-Task Classification with Category Shifts
To generalize to such test data, it is crucial for individual tasks to leverage knowledge from related tasks.
Bayesian Prompt Learning for Image-Language Model Generalization
Our approach regularizes the prompt space, reduces overfitting to the seen prompts and improves the prompt generalization on unseen prompts.
Ranked #1 on
Few-Shot Learning
on UCF101
Fake It Till You Make It: Towards Accurate Near-Distribution Novelty Detection
Effectiveness of our method for both the near-distribution and standard novelty detection is assessed through extensive experiments on datasets in diverse applications such as medical images, object classification, and quality control.
Ranked #2 on
Anomaly Detection
on One-class CIFAR-10
(using extra training data)
Less than Few: Self-Shot Video Instance Segmentation
The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time.
LifeLonger: A Benchmark for Continual Disease Classification
Task and class incremental learning of diseases address the issue of classifying new samples without re-training the models from scratch, while cross-domain incremental learning addresses the issue of dealing with datasets originating from different institutions while retaining the previously obtained knowledge.
Audio-Adaptive Activity Recognition Across Video Domains
This paper strives for activity recognition under domain shift, for example caused by change of scenery or camera viewpoint.
How Do You Do It? Fine-Grained Action Understanding with Pseudo-Adverbs
We aim to understand how actions are performed and identify subtle differences, such as 'fold firmly' vs. 'fold gently'.
Learning to Generalize across Domains on Single Test Samples
We leverage a meta-learning paradigm to learn our model to acquire the ability of adaptation with single samples at training time so as to further adapt itself to each single test sample at test time.
Generative Kernel Continual learning
Kernel continual learning by \citet{derakhshani2021kernel} has recently emerged as a strong continual learner due to its non-parametric ability to tackle task interference and catastrophic forgetting.
Hierarchical Variational Memory for Few-shot Learning Across Domains
To explore and exploit the importance of different semantic levels, we further propose to learn the weights associated with the prototype at each level in a data-driven way, which enables the model to adaptively choose the most generalizable features.
BoxeR: Box-Attention for 2D and 3D Transformers
Specifically, we present BoxeR, short for Box Transformer, which attends to a set of boxes by predicting their transformation from a reference window on an input feature map.
Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation
Most set prediction models in deep learning use set-equivariant operations, but they actually operate on multisets.
Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias
This paper strives to address image classifier bias, with a focus on both feature and label embedding spaces.
Diagnosing Errors in Video Relation Detectors
Video relation detection forms a new and challenging problem in computer vision, where subjects and objects need to be localized spatio-temporally and a predicate label needs to be assigned if and only if there is an interaction between the two.
Social Fabric: Tubelet Compositions for Video Relation Detection
We also propose Social Fabric: an encoding that represents a pair of object tubelets as a composition of interaction primitives.
Skeleton-Contrastive 3D Action Representation Learning
In particular, we propose inter-skeleton contrastive learning, which learns from multiple different input skeleton representations in a cross-contrastive manner.
Feature-Supervised Action Modality Transfer
This paper strives for action recognition and detection in video modalities like RGB, depth maps or 3D-skeleton sequences when only limited modality-specific labeled examples are available.
Kernel Continual Learning
We further introduce variational random features to learn a data-driven kernel for each task.
On Measuring and Controlling the Spectral Bias of the Deep Image Prior
The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it's parameters to reconstruct a single degraded image.
Pruning Edges and Gradients to Learn Hypergraphs from Larger Sets
We address two common scaling problems encountered in set-to-hypergraph tasks that limit the size of the input set: the exponentially growing number of hyperedges and the run-time complexity, both leading to higher memory requirements.
Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation
A critical challenge faced by supervised word sense disambiguation (WSD) is the lack of large annotated datasets with sufficient coverage of words in their diversity of senses.
Unsharp Mask Guided Filtering
The state-of-the-art leverages deep networks to estimate the two core coefficients of the guided filter.
Attentional Prototype Inference for Few-Shot Semantic Segmentation
In this work, we propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot semantic segmentation.
A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data.
MetaKernel: Learning Variational Random Features with Limited Labels
Specifically, we propose learning variational random features in a data-driven manner to obtain task-specific kernels by leveraging the shared knowledge provided by related tasks in a meta-learning setting.
Motion-Augmented Self-Training for Video Recognition at Smaller Scale
With the motion model we generate pseudo-labels for a large unlabeled video collection, which enables us to transfer knowledge by learning to predict these pseudo-labels with an appearance model.
Safe Fakes: Evaluating Face Anonymizers for Face Detectors
Our experiments investigate the suitability of anonymization methods for maintaining face detector performance, the effect of detectors overtraining on anonymization artefacts, dataset size for training an anonymizer, and the effect of training time of anonymization GANs.
Object Priors for Classifying and Localizing Unseen Actions
This work strives for the classification and localization of human actions in videos, without the need for any labeled video training examples.
Few-Shot Transformation of Common Actions into Time and Space
This paper introduces the task of few-shot common action localization in time and space.
TubeR: Tubelet Transformer for Video Action Detection
We propose TubeR: a simple solution for spatio-temporal video action detection.
LiftPool: Bidirectional ConvNet Pooling
Pooling is a critical operation in convolutional neural networks for increasing receptive fields and improving robustness to input variations.
Repetitive Activity Counting by Sight and Sound
We also introduce a variant of this dataset for repetition counting under challenging vision conditions.
MetaNorm: Learning to Normalize Few-Shot Batches Across Domains
Batch normalization plays a crucial role when training deep neural networks.
Variational Invariant Learning for Bayesian Domain Generalization
In the probabilistic modeling framework, we introduce a domain-invariant principle to explore invariance across domains in a unified way.
Learning to Learn Variational Semantic Memory
The variational semantic memory accrues and stores semantic information for the probabilistic inference of class prototypes in a hierarchical Bayesian framework.
Set Prediction without Imposing Structure as Conditional Density Estimation
In this paper, we propose an alternative to training via set losses by viewing learning as conditional density estimation.
Bias-Awareness for Zero-Shot Learning the Seen and Unseen
We propose a bias-aware learner to map inputs to a semantic embedding space for generalized zero-shot learning.
PointMixup: Augmentation for Point Clouds
In this paper, we define data augmentation between point clouds as a shortest path linear interpolation.
Ranked #3 on
3D Point Cloud Data Augmentation
on ModelNet40
3D Point Cloud Classification
3D Point Cloud Data Augmentation
+2
Localizing the Common Action Among a Few Videos
The start and end of an action in a long untrimmed video is determined based on just a hand-full of trimmed video examples containing the same action, without knowing their common class label.
Learning to Learn with Variational Information Bottleneck for Domain Generalization
Domain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift.
Actor-Transformers for Group Activity Recognition
This paper strives to recognize individual actions and group activities from videos.
Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
We propose to enforce semantic consistency at all stages of (generalized) zero-shot learning: training, feature synthesis and classification.
Ranked #1 on
Zero-Shot Learning
on SUN Attribute
Cloth in the Wind: A Case Study of Physical Measurement through Simulation
For many of the physical phenomena around us, we have developed sophisticated models explaining their behavior.
Open Cross-Domain Visual Search
In this paper, we make the step towards an open setting where multiple visual domains are available.
Go with the Flow: Perception-refined Physics Simulation
Nevertheless, inferring specifics from visual observations is challenging due to the high number of causally underlying physical parameters -- including material properties and external forces.
Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation
Few-shot learning is a nascent research topic, motivated by the fact that traditional deep learning methods require tremen- dous amounts of data.
Ranked #1 on
Few-Shot Semantic Segmentation
on Pascal5i
Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres
We observe many continuous output problems in computer vision are naturally contained in closed geometrical manifolds, like the Euler angles in viewpoint estimation or the normals in surface normal estimation.
Cooperative Embeddings for Instance, Attribute and Category Retrieval
These relationships enable them to cooperate for their mutual benefits for image retrieval.
Dance with Flow: Two-in-One Stream Action Detection
With only half the computation and parameters of the state-of-the-art two-stream methods, our two-in-one stream still achieves impressive results on UCF101-24, UCFSports and J-HMDB.
Ranked #2 on
Action Detection
on UCF101-24
(mAP metric)
Counting with Focus for Free
To assist both the density estimation and the focus from segmentation, we also introduce an improved kernel size estimator for the point annotations.
Hyperspherical Prototype Networks
This paper introduces hyperspherical prototype networks, which unify classification and regression with prototypes on hyperspherical output spaces.
Anomaly Locality in Video Surveillance
This paper strives for the detection of real-world anomalies such as burglaries and assaults in surveillance videos.
Pixelated Semantic Colorization
We propose two ways to incorporate object semantics into the colorization model: through a pixelated semantic embedding and a pixelated semantic generator.
Pixel-level Semantics Guided Image Colorization
While many image colorization algorithms have recently shown the capability of producing plausible color versions from gray-scale photographs, they still suffer from the problems of context confusion and edge color bleeding.
Video Time: Properties, Encoders and Evaluation
Time-aware encoding of frame sequences in a video is a fundamental problem in video understanding.
Spatio-Temporal Instance Learning: Action Tubes from Class Supervision
Rather than disconnecting the spatio-temporal learning from the training, we propose Spatio-Temporal Instance Learning, which enables action localization directly from box proposals in video frames.
Multiple Instance Learning
Spatio-Temporal Action Localization
+1
Repetition Estimation
Estimating visual repetition from realistic video is challenging as periodic motion is rarely perfectly static and stationary.
Pointly-Supervised Action Localization
Experimental evaluation on three action localization datasets shows our pointly-supervised approach (i) is as effective as traditional box-supervision at a fraction of the annotation cost, (ii) is robust to sparse and noisy point annotations, (iii) benefits from pseudo-points during inference, and (iv) outperforms recent weakly-supervised alternatives.
Actor and Action Video Segmentation from a Sentence
This paper strives for pixel-level segmentation of actors and their actions in video content.
Ranked #10 on
Referring Expression Segmentation
on J-HMDB
Real-World Repetition Estimation by Div, Grad and Curl
We consider the problem of estimating repetition in video, such as performing push-ups, cutting a melon or playing violin.
The New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval
Over the past decade, emoji have emerged as a new and widespread form of digital communication, spanning diverse social networks and spoken languages.
Predicting Visual Features from Text for Image and Video Caption Retrieval
This paper strives to find amidst a set of sentences the one best describing the content of a given image or video.
Spatial-Aware Object Embeddings for Zero-Shot Localization and Classification of Actions
Action localization and classification experiments on four contemporary action video datasets support our proposal.
Localizing Actions from Video Labels and Pseudo-Annotations
The goal of this paper is to determine the spatio-temporal location of actions in video.
Tracking by Natural Language Specification
This paper strives to track a target object in a video.
Ranked #14 on
Referring Expression Segmentation
on J-HMDB
Searching Scenes by Abstracting Things
Second, we propose translating the things syntax in linguistic abstract statements and study their descriptive effect to retrieve scenes.
Tubelets: Unsupervised action proposals from spatiotemporal super-voxels
First, inspired by selective search for object proposals, we introduce an approach to generate action proposals from spatiotemporal super-voxels in an unsupervised manner, we call them Tubelets.
VideoLSTM Convolves, Attends and Flows for Action Recognition
We present a new architecture for end-to-end sequence learning of actions in video, we call VideoLSTM.
Spot On: Action Localization from Pointly-Supervised Proposals
Rather than annotating boxes, we propose to annotate actions in video with points on a sparse subset of frames only.
Word2VisualVec: Image and Video to Sentence Matching by Visual Feature Prediction
This paper strives to find the sentence best describing the content of an image or video.
The ImageNet Shuffle: Reorganized Pre-training for Video Event Detection
To deal with the problems of over-specific classes and classes with few images, we introduce a bottom-up and top-down approach for reorganization of the ImageNet hierarchy based on all its 21, 814 classes and more than 14 million images.
VideoStory Embeddings Recognize Events when Examples are Scarce
In our proposed embedding, which we call VideoStory, the correlations between the terms are utilized to learn a more effective representation by optimizing a joint objective balancing descriptiveness and predictability. We show how learning the VideoStory using a multimodal predictability loss, including appearance, motion and audio features, results in a better predictable representation.
Objects2action: Classifying and localizing actions without any video example
Our key contribution is objects2action, a semantic word embedding that is spanned by a skip-gram model of thousands of object categories.
Ranked #14 on
Zero-Shot Action Recognition
on UCF101
No Spare Parts: Sharing Part Detectors for Image Categorization
This work aims for image categorization using a representation of distinctive parts.
TagBook: A Semantic Video Representation without Supervision for Event Detection
We consider the problem of event detection in video for scenarios where only few, or even zero examples are available for training.
Active Transfer Learning with Zero-Shot Priors: Reusing Past Datasets for Future Tasks
How can we reuse existing knowledge, in the form of available datasets, when solving a new and apparently unrelated target task from a set of unlabeled data?
What do 15,000 Object Categories Tell Us About Classifying and Localizing Actions?
This paper contributes to automatic classification and localization of human actions in video.
Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval
Where previous reviews on content-based image retrieval emphasize on what can be seen in an image to bridge the semantic gap, this survey considers what people tag about an image.
Action Localization with Tubelets from Motion
Our approach significantly outperforms the state-of-the-art on both datasets, while restricting the search of actions to a fraction of possible bounding box sequences.
Fisher and VLAD with FLAIR
Finally, by multiple codeword assignments, we achieve exact and approximate Fisher vectors with FLAIR.
Locality in Generic Instance Search from One Example
This paper aims for generic instance search from a single example.
COSTA: Co-Occurrence Statistics for Zero-Shot Classification
In this paper we aim for zero-shot classification, that is visual recognition of an unseen class by using knowledge transfer from known classes.
