Search Results for author:
Found 167 papers, 65 papers with code
Count- and Similarity-aware R-CNN for Pedestrian Detection
We further introduce a count-and-similarity branch within the two-stage detection framework, which predicts pedestrian count as well as proposal similarity.
Multi-view Action Recognition using Cross-view Video Prediction
We evaluate the effectiveness of the learned representation for multi-view video action recognition in a supervised approach.
Learning Situation Hyper-Graphs for Video Question Answering
The proposed method is trained in an end-to-end manner and optimized by a VQA loss with the cross-entropy function and a Hungarian matching loss for the situation graph prediction.
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting
Through this prompting scheme, we can achieve state-of-the-art zero-shot performance on Kinetics-600, HMDB51 and UCF101 while remaining competitive in the supervised setting.
$R^{2}$Former: Unified $R$etrieval and $R$eranking Transformer for Place Recognition
Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database.
Video Instance Segmentation in an Open-World
Open-world formulation relaxes the close-world static-learning assumption as follows: (a) first, it distinguishes a set of known categories as well as labels an unknown object as `unknown' and then (b) it incrementally learns the class of an unknown as and when the corresponding semantic labels become available.
Diffusion Action Segmentation
Temporal action segmentation is crucial for understanding long-form videos.
Ranked #1 on
Action Segmentation
on Breakfast
TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition
We observe that these representations complement each other depending on the nature of the action.
3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers
Accurate 3D mitochondria instance segmentation in electron microscopy (EM) is a challenging problem and serves as a prerequisite to empirically analyze their distributions and morphology.
Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes
To this end, we introduce an end-to-end transformer-based architecture that exploits the relationship between different geographic levels (which we refer to as hierarchies) and the corresponding visual scene information in an image through hierarchical cross-attention.
PivoTAL: Prior-Driven Supervision for Weakly-Supervised Temporal Action Localization
To address this, we present PivoTAL, Prior-driven Supervision for Weakly-supervised Temporal Action Localization, to approach WTAL from a localization-by-localization perspective by learning to localize the action snippets directly.
Weakly Supervised Action Localization
Weakly Supervised Temporal Action Localization
Class Prototypes Based Contrastive Learning for Classifying Multi-Label and Fine-Grained Educational Videos
We learn a class prototype for each class and a loss function is employed to minimize the distances between a class prototype and the samples from the class.
R2Former: Unified Retrieval and Reranking Transformer for Place Recognition
Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database.
When Do Curricula Work in Federated Learning?
Specifically, we aim to investigate how ordered learning principles can contribute to alleviating the heterogeneity effects in FL.
Lightning Fast Video Anomaly Detection via Adversarial Knowledge Distillation
We propose a very fast frame-level model for anomaly detection in video, which learns to detect anomalies by distilling knowledge from multiple highly accurate object-level teacher models.
Query Efficient Cross-Dataset Transferable Black-Box Attack on Action Recognition
By using a nearly disjoint dataset to train the substitute model, our method removes the requirement that the substitute model be trained using the same dataset as the target model, and leverages queries to the target model to retain the fooling rate benefits provided by query-based methods.
Person Image Synthesis via Denoising Diffusion Model
In this work, we show how denoising diffusion models can be applied for high-fidelity person image synthesis with strong sample diversity and enhanced mode coverage of the learnt data distribution.
3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds
We propose a method for joint detection and tracking of multiple objects in 3D point clouds, a task conventionally treated as a two-step process comprising object detection followed by data association.
Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future
In this paper, we review adversarial pretraining of self-supervised deep networks including both convolutional neural networks and vision transformers.
TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos
Drone-to-drone detection using visual feed has crucial applications like avoiding collision with other drones/airborne objects, tackling a drone attack or coordinating flight with other drones.
Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks
Though successful, federated learning presents new challenges for machine learning, especially when the issue of data heterogeneity, also known as Non-IID data, arises.
Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection
In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, as well as a transformer for channel-wise attention.
Ranked #4 on
Anomaly Detection
on CUHK Avenue
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces
This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters.
Diffusion Models in Vision: A Survey
Denoising diffusion models represent a recent emerging topic in computer vision, demonstrating remarkable results in the area of generative modeling.
Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility
We establish that this is a result of the presence of false negative pairs in the training process, which increases model sensitivity to input perturbations.
SSMTL++: Revisiting Self-Supervised Multi-Task Learning for Video Anomaly Detection
A self-supervised multi-task learning (SSMTL) framework for video anomaly detection was recently introduced in literature.
Ranked #2 on
Anomaly Detection
on CUHK Avenue
GAMa: Cross-view Video Geo-localization
There are no existing datasets for this problem, therefore we propose GAMa dataset, a large-scale dataset with ground videos and corresponding aerial images.
Towards Realistic Semi-Supervised Learning
We also highlight the flexibility of our approach in solving novel class discovery task, demonstrate its stability in dealing with imbalanced data, and complement our approach with a technique to estimate the number of novel classes
Ranked #1 on
Open-World Semi-Supervised Learning
on ImageNet-100
(Novel accuracy (10% Labeled) metric)
Novel Class Discovery
Open-World Semi-Supervised Learning
+1
Weakly Supervised Grounding for VQA in Vision-Language Transformers
Transformers for visual-language representation learning have been getting a lot of interest and shown tremendous performance on visual question answering (VQA) and grounding.
OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning
In the open-world SSL problem, the objective is to recognize samples of known classes, and simultaneously detect and cluster samples belonging to novel classes present in unlabeled data.
Ranked #1 on
Open-World Semi-Supervised Learning
on CIFAR-10
Self-Supervised Learning for Videos: A Survey
In this survey, we provide a review of existing approaches on self-supervised learning focusing on the video domain.
Learning with Capsules: A Survey
The aim of this survey is to provide a comprehensive overview of the capsule network research landscape, which will serve as a valuable resource for the community going forward.
EBM Life Cycle: MCMC Strategies for Synthesis, Defense, and Density Modeling
This work presents strategies to learn an Energy-Based Model (EBM) according to the desired length of its MCMC sampling trajectories.
Tag-Based Attention Guided Bottom-Up Approach for Video Instance Segmentation
Video Instance Segmentation is a fundamental computer vision task that deals with segmenting and tracking object instances across a video sequence.
Self-Supervised Video Object Segmentation via Cutout Prediction and Tagging
We propose a novel self-supervised Video Object Segmentation (VOS) approach that strives to achieve better object-background discriminability for accurate object segmentation.
Video Action Detection: Analysing Limitations and Challenges
Beyond possessing large enough size to feed data hungry machines (eg, transformers), what attributes measure the quality of a dataset?
PSTR: End-to-End One-Step Person Search With Transformers
We propose a novel one-step transformer-based person search framework, PSTR, that jointly performs person detection and re-identification (re-id) in a single architecture.
TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization
It does not rely on polar transform and infers faster than CNN-based methods.
SPAct: Self-supervised Privacy Preservation for Action Recognition
Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset.
Ranked #1 on
Action Classification
on UCF101
UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning
To combat label noise, recent state-of-the-art methods employ some sort of sample selection mechanism to select a possibly clean subset of data.
Mesh Convolution with Continuous Filters for 3D Surface Parsing
In this paper, we propose a series of modular operations for effective geometric feature learning from 3D triangle meshes.
OW-DETR: Open-world Detection Transformer
In the case of incremental object detection, OW-DETR outperforms the state-of-the-art for all settings on PASCAL VOC.
Routing with Self-Attention for Multimodal Capsule Networks
We present a new multimodal capsule network that allows us to leverage the strength of capsules in the context of a multimodal learning framework on large amounts of video data.
Reformulating Zero-shot Action Recognition for Multi-label Actions
Given a video and a set of action classes, our method predicts a set of confidence scores for each class independently.
Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection
Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field.
Ranked #1 on
Anomaly Detection
on CUHK Avenue
(TBDC metric)
UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection
This is a closed-set scenario that fails to test the capability of systems at detecting new anomaly types.
Ranked #5 on
Anomaly Detection
on CUHK Avenue
(using extra training data)
"Knights": First Place Submission for VIPriors21 Action Recognition Challenge at ICCV 2021
This technical report presents our approach "Knights" to solve the action recognition task on a small subset of Kinetics-400 i. e. Kinetics400ViPriors without using any extra-data.
Discriminative Region-based Multi-Label Zero-Shot Learning
We note that the best existing multi-label ZSL method takes a shared approach towards attending to region features with a common set of attention maps for all the classes.
Ranked #2 on
Multi-label zero-shot learning
on Open Images V4
Advances in adversarial attacks and defenses in computer vision: A survey
In [2], we reviewed the contributions made by the computer vision community in adversarial attacks on deep learning (and their defenses) until the advent of year 2018.
Video Generation from Text Employing Latent Path Construction for Temporal Modeling
To the best of our knowledge, this is the very first work on the text (free-form sentences) to video generation on more realistic video datasets like Actor and Action Dataset (A2D) or UCF101.
TinyAction Challenge: Recognizing Real-world Low-resolution Activities in Videos
While various approaches have been shown effective for recognition task in recent works, they often do not deal with videos of lower resolution where the action is happening in a tiny region.
VisDrone-CC2020: The Vision Meets Drone Crowd Counting Challenge Results
Crowd counting on the drone platform is an interesting topic in computer vision, which brings new challenges such as small object inference, background clutter and wide viewpoint.
Controlled Caption Generation for Images Through Adversarial Attacks
In contrast, we propose a GAN-based algorithm for crafting adversarial examples for neural image captioning that mimics the internal representation of the CNN such that the resulting deep features of the input image enable a controlled incorrect caption generation through the recurrent network.
Florida Wildlife Camera Trap Dataset
Trail camera imagery has increasingly gained popularity amongst biologists for conservation and ecological research.
Out-of-Distribution Detection Using Union of 1-Dimensional Subspaces
In this paper, we argue that OOD samples can be detected more easily if the training data is embedded into a low-dimensional space, such that the embedded training samples lie on a union of 1-dimensional subspaces.
Novel View Video Prediction Using a Dual Representation
We address the problem of novel view video prediction; given a set of input video clips from a single/multiple views, our network is able to predict the video from a novel view.
PC-DAN: Point Cloud based Deep Affinity Network for 3D Multi-Object Tracking (Accepted as an extended abstract in JRDB-ACT Workshop at CVPR21)
In recent times, the scope of LIDAR (Light Detection and Ranging) sensor-based technology has spread across numerous fields.
PLM: Partial Label Masking for Imbalanced Multi-label Classification
By stochastically masking labels during loss computation, the method balances this ratio for each class, leading to improved recall on minority classes and improved precision on frequent classes.
MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurations
MutualNet is a general training methodology that can be applied to various network structures (e. g., 2D networks: MobileNets, ResNet, 3D networks: SlowFast, X3D) and various tasks (e. g., image classification, object detection, segmentation, and action recognition), and is demonstrated to achieve consistent improvements on a variety of datasets.
Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules
In this paper, we focus on a more relaxed setting: the grounding of relevant visual entities in a weakly supervised manner by training on the VQA task alone.
Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities
This paper proposes a novel approach for unsupervised sub-action learning in complex activities.
Ranked #24 on
Action Segmentation
on Breakfast
Handwriting Transformers
We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns.
Dogfight: Detecting Drones from Drones Videos
The erratic movement of the source and target drones, small size, arbitrary shape, large intensity variations, and occlusion make this problem quite challenging.
LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack
It is set forth that the proposed sparse perturbation is the most aligned sparse perturbation with the shortest path from the input sample to the decision boundary for some initial adversarial sample (the best sparse approximation of shortest path, likely to fool the model).
Modeling Multi-Label Action Dependencies for Temporal Action Localization
We propose to improve action localization performance by modeling these action dependencies in a novel attention-based Multi-Label Action Dependency (MLAD)layer.
Ranked #1 on
Action Detection
on Multi-THUMOS
Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning
Equivariance or invariance has been employed standalone in the previous works; however, to the best of our knowledge, they have not been used jointly.
TCLR: Temporal Contrastive Learning for Video Representation
However, prior work on contrastive learning for video data has not explored the effect of explicitly encouraging the features to be distinct across the temporal dimension.
Ranked #9 on
Self-supervised Video Retrieval
on UCF101
In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
The recent research in semi-supervised learning (SSL) is mostly dominated by consistency regularization based methods which achieve strong performance.
Transformers in Vision: A Survey
Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems.
Video Geo-Localization Employing Geo-Temporal Feature Learning and GPS Trajectory Smoothing
In this paper, we address the problem of video geo-localization by proposing a Geo-Temporal Feature Learning (GTFL) Network to simultaneously learn the discriminative features between the query videos and gallery images for estimating the geo-spatial trajectory of a query video.
Face Image Retrieval With Attribute Manipulation
For example, a user can ask for retrieving images similar to a query image, but with a different hair color, and no preference for absence/presence of eyeglasses in the results.
Asymptotic Optimality of Self-Representative Low-Rank Approximation and Its Applications
The approach is based on the concept of {\em self-rank}, defined as the minimum number of samples needed to reconstruct all samples with an accuracy proportional to the rank-$K$ approximation.
Deep Learning-Based Human Pose Estimation: A Survey
Furthermore, 2D and 3D human pose estimation datasets and evaluation metrics are included.
Correct block-design experiments mitigate temporal correlation bias in EEG classification
We argue that the reason why Li et al. [1] observe such high correlation in EEG data is their unconventional experimental design and settings that violate the basic cognitive neuroscience design recommendations, first and foremost the one of limiting the experiments' duration, as instead done in [2].
Anomaly Detection in Video via Self-Supervised and Multi-Task Learning
To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture.
Ranked #2 on
Anomaly Detection
on UCSD Peds2
Abnormal Event Detection In Video
Anomaly Detection In Surveillance Videos
+4
MMFT-BERT: Multimodal Fusion Transformer with BERT Encodings for Visual Question Answering
We present MMFT-BERT(MultiModal Fusion Transformer with BERT encodings), to solve Visual Question Answering (VQA) ensuring individual and combined processing of multiple input modalities.
Meta-learning the Learning Trends Shared Across Tasks
This demonstrates their ability to acquire transferable knowledge, a capability that is central to human learning.
Uncertainty Estimation and Sample Selection for Crowd Counting
We present sample selection strategies which make use of the density and uncertainty of predictions from the networks trained on one domain to select the informative images from a target domain of interest to acquire human annotation.
A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video
Following the standard formulation of abnormal event detection as outlier detection, we propose a background-agnostic framework that learns from training videos containing only normal events.
Abnormal Event Detection In Video
Anomaly Detection In Surveillance Videos
+2
Simultaneous Detection and Tracking with Motion Modelling for Multiple Object Tracking
Deep learning-based Multiple Object Tracking (MOT) currently relies on off-the-shelf detectors for tracking-by-detection. This results in deep models that are detector biased and evaluations that are detector influenced.
Deep Photo Cropper and Enhancer
In the photo enhancer, we employ super-resolution to increase the number of pixels in the embedded image and reduce the effect of stretching and distortion of pixels.
Cassandra: Detecting Trojaned Networks from Adversarial Perturbations
We also propose an anomaly detection method to identify the target class in a Trojaned network.
Ranked #1 on
Adversarial Defense
on TrojAI Round 1
Odyssey: Creation, Analysis and Detection of Trojan Models
We propose a detector that is based on the analysis of the intrinsic DNN properties; that are affected due to the Trojaning process.
TinyVIRAT: Low-resolution Video Action Recognition
In real-world surveillance environments, the actions in videos are captured at a wide range of resolutions.
Self-supervised Knowledge Distillation for Few-shot Learning
Our experiments show that, even in the first stage, self-supervision can outperform current state-of-the-art methods, with further gains achieved by our second stage distillation process.
Ranked #12 on
Few-Shot Image Classification
on FC100 5-way (5-shot)
Text Synopsis Generation for Egocentric Videos
We Input a set of video shots and the network generates a text description for each shot.
Gabriella: An Online System for Real-Time Activity Detection in Untrimmed Security Videos
For tubelet extraction, we propose a localization network which takes a video clip as input and spatio-temporally detects potential foreground regions at multiple scales to generate action tubelets.
RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery
Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity.
Adversarial Learning for Personalized Tag Recommendation
We demonstrate the effectiveness of the proposed model on two different large-scale and publicly available datasets, YFCC100M and NUS-WIDE.
iTAML: An Incremental Task-Agnostic Meta-learning Approach
In this paper, we hypothesize this problem can be avoided by learning a set of generalized parameters, that are neither specific to old nor new tasks.
Subspace Capsule Network
In this paper, we propose the SubSpace Capsule Network (SCN) that exploits the idea of capsule networks to model possible variations in the appearance or implicitly defined properties of an entity through a group of capsule subspaces instead of simply grouping neurons to create capsules.
On Symbiosis of Attribute Prediction and Semantic Segmentation
In SSG, the same idea is applied to the intermediate layers of the network.
Human Action Recognition in Drone Videos using a Few Aerial Training Examples
However, using deep neural networks for automatic aerial action recognition is difficult due to the need for a large number of training aerial human action videos.
CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing
In this work we propose a capsule-based approach for semi-supervised video object segmentation.
One-shot visual object segmentation
Optical Flow Estimation
+3
An Efficient 3D CNN for Action/Object Segmentation in Video
Convolutional Neural Network (CNN) based image segmentation has made great progress in recent years.
Ranked #62 on
Semi-Supervised Video Object Segmentation
on DAVIS 2016
Deep Constrained Dominant Sets for Person Re-identification
By optimizing the constrained clustering in an end-to-end manner, we naturally leverage the contextual knowledge of a set of images corresponding to the given person-images.
Ranked #2 on
Person Re-Identification
on CUHK03
(Rank-5 metric)
Bridging the Domain Gap for Ground-to-Aerial Image Matching
Our Feature Fusion method combines the complementary features from a synthesized aerial image with the corresponding ground features to obtain a robust query representation.
Crowd Transformer Network
Most of the existing crowd counting approaches rely on local features for estimating the crowd density map.
EgoReID Dataset: Person Re-identification in Videos Acquired by Mobile Devices with First-Person Point-of-View
In recent years, we have seen the performance of video-based person Re-Identification (ReID) methods have improved considerably.
Multi-modal Capsule Routing for Actor and Action Video Segmentation Conditioned on Natural Language Queries
The existing works on actor-action localization are mainly focused on localization in a single frame instead of the full video.
Iterative Projection and Matching: Finding Structure-preserving Representatives and Its Application to Computer Vision
In our algorithm, at each iteration, the maximum information from the structure of the data is captured by one selected sample, and the captured information is neglected in the next iterations by projection on the null-space of previously selected samples.
Unsupervised Meta-Learning For Few-Shot Image Classification
In this paper, we propose UMTRA, an algorithm that performs unsupervised, model-agnostic meta-learning for classification tasks.
Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning
We demonstrate the effectiveness of the proposed method in rendering view-aware as well as time-aware video clips on two different real-world datasets including UCF-101 and NTU-RGB+D.
Deep Affinity Network for Multiple Object Tracking
In this paper, we harness the power of deep learning for data association in tracking by jointly modelling object appearances and their affinities between different frames in an end-to-end fashion.
Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features
After verifying that visual information can be extracted from EEG data, we introduce a multimodal approach that uses deep image and EEG encoders, trained in a siamese configuration, for learning a joint manifold that maximizes a compatibility measure between visual features and brain representations.
Pay attention! - Robustifying a Deep Visuomotor Policy through Task-Focused Attention
In this paper, we propose an approach for augmenting a deep visuomotor policy trained through demonstrations with Task Focused visual Attention (TFA).
Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds
With multiple crowd gatherings of millions of people every year in events ranging from pilgrimages to protests, concerts to marathons, and festivals to funerals; visual crowd analysis is emerging as a new frontier in computer vision.
Ranked #7 on
Crowd Counting
on UCF-QNRF
Training Faster by Separating Modes of Variation in Batch-normalized Models
We show that assuming samples within a mini-batch are from the same probability density function, then BN is identical to the Fisher vector of a Gaussian distribution.
Video Description: A Survey of Methods, Datasets and Evaluation Metrics
Video description is the automatic generation of natural language sentences that describe the contents of a given video.
VideoCapsuleNet: A Simplified Network for Action Detection
In this work, we present a more elegant solution for action detection based on the recently developed capsule network.
Task-Agnostic Meta-Learning for Few-shot Learning
Meta-learning approaches have been proposed to tackle the few-shot learning problem. Typically, a meta-learner is trained on a variety of tasks in the hopes of being generalizable to new tasks.
Norm-Preservation: Why Residual Networks Can Become Extremely Deep?
We prove that the skip connections in the residual blocks facilitate preserving the norm of the gradient, and lead to stable back-propagation, which is desirable from optimization perspective.
Human Semantic Parsing for Person Re-identification
In this paper, we propose to adopt human semantic parsing which, due to its pixel-level accuracy and capability of modeling arbitrary contours, is naturally a better alternative.
Ranked #73 on
Person Re-Identification
on Market-1501
Real-world Anomaly Detection in Surveillance Videos
To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labeled training videos, i. e. the training labels (anomalous or normal) are at video-level instead of clip-level.
Ranked #2 on
Abnormal Event Detection In Video
on UBI-Fights
Activity Recognition
Anomaly Detection In Surveillance Videos
+2
Visual Text Correction
A semantic inconsistency between the sentence and the video or between the words of a sentence can result in an inaccurate description.
An End-to-end 3D Convolutional Neural Network for Action Detection and Segmentation in Videos
A video is first divided into equal length clips and next for each clip a set of tube proposals are generated based on 3D CNN features.
Generative Adversarial Networks Conditioned by Brain Signals
In this work, we build on the latter class of approaches and investigate the possibility of driving and conditioning the image generation process by means of brain signals recorded, through an electroencephalograph (EEG), while users look at images from a set of 40 ImageNet object categories with the objective of generating the seen images.
Unsupervised Action Discovery and Localization in Videos
Once classes are discovered, training videos within each cluster are selected to perform automatic spatio-temporal annotations, by first oversegmenting videos in each discovered class into supervoxels and constructing a directed graph to apply a variant of knapsack problem with temporal constraints.
Semi Supervised Semantic Segmentation Using Generative Adversarial Network
Semantic segmentation has been a long standing challenging task in computer vision.
Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets
In this paper, a unified three-layer hierarchical approach for solving tracking problems in multiple non-overlapping cameras is proposed.
Improving Facial Attribute Prediction using Semantic Segmentation
We build our facial attribute prediction model jointly with a deep semantic segmentation network.
Video Fill In the Blank using LR/RL LSTMs with Spatial-Temporal Attentions
Since the source sentence is broken into two fragments: the sentence's left fragment (before the blank) and the sentence's right fragment (after the blank), traditional Recurrent Neural Networks cannot encode this structure accurately because of many possible variations of the missing word in terms of the location and type of the word in the source sentence.
ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information
To reduce the large search space, the first stage (ClusterNet) takes in a set of extremely large video frames, combines the motion and appearance information within the convolutional architecture, and proposes regions of objects of interest (ROOBI).
Unsupervised Action Proposal Ranking through Proposal Recombination
Given the action proposals in a video, the goal of the proposed work is to generate a few better action proposals that are ranked properly.
Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos
A video is first divided into equal length clips and for each clip a set of tube proposals are generated next based on 3D Convolutional Network (ConvNet) features.
Ranked #3 on
Action Detection
on UCF101-24
Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
Semantic segmentation has been a long standing challenging task in computer vision.
Cross-View Image Matching for Geo-localization in Urban Environments
Next, for each building in the query image, we retrieve the $k$ nearest neighbors from the reference buildings using a Siamese network trained on both positive matching image pairs and negative pairs.
Cross-View Image-to-Image Translation
Image Classification
+2
Large-scale Image Geo-Localization Using Dominant Sets
We cast the geo-localization as a clustering problem on local image features.
Re-identification of Humans in Crowds using Personal, Social and Environmental Constraints
This paper addresses the problem of human re-identification across non-overlapping cameras in crowds. Re-identification in crowded scenes is a challenging problem due to large number of people and frequent occlusions, coupled with changes in their appearance due to different properties and exposure of cameras.
Online Localization and Prediction of Actions and Interactions
For online prediction of action (interaction) confidences, we propose an approach based on Structural SVM that operates on short video segments, and is trained with the objective that confidence of an action or interaction increases as time progresses.
On Duality Of Multiple Target Tracking and Segmentation
For segmentation, multi-label Conditional Random Field (CRF) is applied to a superpixel based spatio-temporal graph in a segment of video to assign background or target labels to every superpixel.
Video Fill in the Blank with Merging LSTMs
In the experiments, we have demonstrated the superior performance of the proposed method on the challenging "Movie Fill-in-the-Blank" dataset.
Deep Learning Human Mind for Automated Visual Classification
In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories.
Scene Labeling Through Knowledge-Based Rules Employing Constrained Integer Linear Programing
In this paper, we propose to use high-level knowledge regarding rules in the inference to incorporate dependencies among regions in the image to improve scores of classification.
Query-Focused Extractive Video Summarization
The decision to include a shot in the summary depends on the shot's relevance to the user query and importance in the context of the video, jointly.
Covariance of Motion and Appearance Featuresfor Spatio Temporal Recognition Tasks
Using an over-complete dictionary of the covariance based descriptors built from labeled training samples, we formulate low-level event recognition as a sparse linear approximation problem.
Predicting the Where and What of Actors and Actions Through Online Action Localization
This paper proposes a novel approach to tackle the challenging problem of 'online action localization' which entails predicting actions and their locations as they happen in a video.
What If We Do Not Have Multiple Videos of the Same Action? -- Video Action Localization Using Web Images
%We reconstruct video action proposals from image action proposals while enforcing consistency across coefficient vectors of multiple frames by consensus regularization.
Optical Flow Estimation
Spatio-Temporal Action Localization
+1
Scene Labeling Using Sparse Precision Matrix
To do this, we formulate the problem as an energy minimization over a graph, whose structure is captured by applying sparse constraint on the elements of the precision matrix.
Fast Zero-Shot Image Tagging
The well-known word analogy experiments show that the recent word vectors capture fine-grained linguistic regularities in words by linear vector offsets, but it is unclear how well the simple vector offsets can encode visual regularities over words.
Ranked #5 on
Multi-label zero-shot learning
on Open Images V4
Automatic Action Annotation in Weakly Labeled Videos
The output of our method is the most action representative proposals from each video.
A Framework for Human Pose Estimation in Videos
A sequence of the best poses is inferred from the abstract body part tracklets through the tree-based optimization.
The THUMOS Challenge on Action Recognition for Videos "in the Wild"
Additionally, we include a comprehensive empirical study evaluating the differences in action recognition between trimmed and untrimmed videos, and how well methods trained on trimmed videos generalize to untrimmed videos.
Binary Quadratic Programing for Online Tracking of Hundreds of People in Extremely Crowded Scenes
In this paper, we propose a tracker that addresses the aforementioned problems and is capable of tracking hundreds of people efficiently.
Autonomous navigation for low-altitude UAVs in urban areas
In recent years, consumer Unmanned Aerial Vehicles have become very popular, everyone can buy and fly a drone without previous experience, which raises concern in regards to regulations and public safety.
Learning a Deep Model for Human Action Recognition from Novel Viewpoints
The strength of our technique is that we learn a single R-NKTM for all actions and all viewpoints for knowledge transfer of any real human action video without the need for re-training or fine-tuning the model.
Human Pose Estimation in Videos
Using the idea of `Association', the optimal tracklets are generated for each abstract body part, in order to enforce the spatiotemporal constraints between body parts in adjacent frames.
Action Localization in Videos Through Context Walk
Context relations are learned during training which capture displacements from all the supervoxels in a video to those belonging to foreground actions.
Target Identity-Aware Network Flow for Online Multiple Target Tracking
In this paper we show that multiple object tracking (MOT) can be formulated in a framework, where the detection and data-association are performed simultaneously.
Geo-Semantic Segmentation
In this paper, we propose a method which leverages information acquired from GIS databases to perform semantic segmentation of the image alongside with geo-referencing each semantic segment with its address and geo-location.
GMMCP Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking
Data association is the backbone to many multiple object tracking (MOT) methods.
Understanding Trajectory Behavior: A Motion Pattern Approach
In the second phase, via a Kmeans clustering approach, we create motion components by clustering the flow vectors with respect to their location and velocity.
Improving Semantic Concept Detection through the Dictionary of Visually-distinct Elements
A video captures a sequence and interactions of concepts that can be static, for instance, objects or scenes, or dynamic, such as actions.
GPS-Tag Refinement using Random Walks with an Adaptive Damping Factor
We develop a robust method for identification and refinement of this subset using the rest of the images in the dataset.
NMF-KNN: Image Annotation using Weighted Multi-view Non-negative Matrix Factorization
Such models become obsolete and require relearning when new images and tags are added to database.
Who Do I Look Like? Determining Parent-Offspring Resemblance via Gated Autoencoders
Recent years have seen a major push for face recognition technology due to the large expansion of image sharing on social networks.
Recognition of Complex Events: Exploiting Temporal Dynamics between Underlying Concepts
While approaches based on bags of features excel at low-level action classification, they are ill-suited for recognizing complex events in video, where concept-based temporal representations currently dominate.
Video Classification using Semantic Concept Co-occurrences
We address the problem of classifying complex videos based on their content.
Face Verification Using Boosted Cross-Image Features
In contrast, we propose to extract cross-image features, i. e. features across the pair of images, which, as we demonstrate, is more discriminative to the similarity and the dissimilarity of faces.
Improving an Object Detector and Extracting Regions Using Superpixels
In general, our method takes detection bounding boxes of a generic detector as input and generates the detection output with higher average precision and precise object regions.
Spatiotemporal Deformable Part Models for Action Detection
Deformable part models have achieved impressive performance for object detection, even on difficult image datasets.
Face Recognition in Movie Trailers via Mean Sequence Sparse Representation-Based Classification
A straightforward application of the popular n-minimization for face recognition on a frame-by-frame basis is prohibitively expensive, so we propose a novel algorithm Mean Sequence SRC (MSSRC) that performs video face recognition using a joint optimization leveraging all of the available video data and the knowledge that the face track frames belong to the same individual.
Multi-source Multi-scale Counting in Extremely Dense Crowd Images
Instead, our approach relies on multiple sources such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals, in an image region.
Ranked #14 on
Crowd Counting
on UCF-QNRF
Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions
The proposed approach has several contributions: First, a novel layered Directed Acyclic Graph (DAG) based framework is presented for detection and segmentation of the primary object in video.
Semi-supervised Learning of Feature Hierarchies for Object Detection in a Video
In order to learn discriminative and compact features, we propose a new feature learning method using a deep neural network based on auto encoders.
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
To the best of our knowledge, UCF101 is currently the most challenging dataset of actions due to its large number of classes, large number of clips and also unconstrained nature of such clips.
Ranked #5 on
Action Recognition In Videos
on UCF101
Action Recognition In Videos
Skeleton Based Action Recognition
+1
