Search Results for author:
Found 38 papers, 22 papers with code
End-to-End Spatio-Temporal Action Localisation with Video Transformers
The most performant spatio-temporal action localisation models use external person proposals and complex external memory banks.
VicTR: Video-conditioned Text Representations for Activity Recognition
All such recipes rely on augmenting visual embeddings with temporal information (i. e., image -> video), often keeping text embeddings unchanged or even being discarded.
Ranked #7 on
Zero-Shot Action Recognition
on HMDB51
CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation
However, the problem of transferring these capabilities learned from image-level supervision to the pixel-level task of segmentation and addressing arbitrary unseen categories at inference makes this task challenging.
Scaling Vision Transformers to 22 Billion Parameters
The scaling of Transformers has driven breakthrough capabilities for language models.
Ranked #1 on
Linear-Probe Classification
on ImageNet
(using extra training data)
Adaptive Computation with Elastic Input Sequence
However, most standard neural networks have the same function type and fixed computation budget on different samples regardless of their nature and difficulty.
How Can Objects Help Action Recognition?
In this paper, we investigate how we can use knowledge of objects to design better video models, namely to process fewer tokens and to improve recognition accuracy.
Audiovisual Masked Autoencoders
Can we leverage the audiovisual information already present in video to improve self-supervised representation learning?
Token Turing Machines
The model's memory module ensures that a new observation will only be processed with the contents of the memory (and not the entire history), meaning that it can efficiently process long sequences with a bounded computational cost at each step.
Ranked #1 on
Action Detection
on Charades
Dynamic Graph Message Passing Networks for Visual Recognition
A fully-connected graph, such as the self-attention operation in Transformers, is beneficial for such modelling, however, its computational overhead is prohibitive.
Beyond Transfer Learning: Co-finetuning for Action Localisation
Transfer learning is the predominant paradigm for training deep networks on small target datasets.
M&M Mix: A Multimodal Multiview Transformer Ensemble
This report describes the approach behind our winning solution to the 2022 Epic-Kitchens Action Recognition Challenge.
Ranked #1 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Simple Open-Vocabulary Object Detection with Vision Transformers
Combining simple architectures with large-scale pre-training has led to massive improvements in image classification.
Ranked #1 on
One-Shot Object Detection
on COCO
Learning with Neighbor Consistency for Noisy Labels
Recent advances in deep learning have relied on large, labelled datasets to train high-capacity models.
End-to-end Generative Pretraining for Multimodal Video Captioning
Recent video and language pretraining frameworks lack the ability to generate sentences.
Ranked #8 on
Video Captioning
on MSR-VTT
(using extra training data)
Multiview Transformers for Video Recognition
Video understanding requires reasoning at multiple spatiotemporal resolutions -- from short fine-grained motions to events taking place over longer durations.
Ranked #2 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
TokenLearner: Adaptive Space-Time Tokenization for Videos
In this paper, we introduce a novel visual representation learning which relies on a handful of adaptively learned tokens, and which is applicable to both image and video understanding tasks.
PolyViT: Co-training Vision Transformers on Images, Videos and Audio
Can we train a single transformer model capable of processing multiple modalities and datasets, whilst sharing almost all of its learnable parameters?
SCENIC: A JAX Library for Computer Vision Research and Beyond
Scenic is an open-source JAX library with a focus on Transformer-based models for computer vision research and beyond.
Compressive Visual Representations
We verify this by developing SimCLR and BYOL formulations compatible with the Conditional Entropy Bottleneck (CEB) objective, allowing us to both measure and control the amount of compression in the learned representation, and observe their impact on downstream tasks.
Ranked #29 on
Self-Supervised Image Classification
on ImageNet
Attention Bottlenecks for Multimodal Fusion
Humans perceive the world by concurrently processing and fusing high-dimensional inputs from multiple modalities such as vision and audio.
Ranked #1 on
Audio Classification
on VGGSound
(Top 5 Accuracy metric)
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?
In this paper, we introduce a novel visual representation learning which relies on a handful of adaptively learned tokens, and which is applicable to both image and video understanding tasks.
Ranked #1 on
Action Classification
on Charades
Unified Graph Structured Models for Video Understanding
Accurate video understanding involves reasoning about the relationships between actors, objects and their environment, often over long temporal intervals.
ViViT: A Video Vision Transformer
We present pure-transformer based models for video classification, drawing upon the recent success of such models in image classification.
Ranked #8 on
Action Classification
on Moments in Time
(Top 5 Accuracy metric, using extra
training data)
Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos
Despite the recent advances in video classification, progress in spatio-temporal action recognition has lagged behind.
Dual Graph Convolutional Network for Semantic Segmentation
Exploiting long-range contextual information is key for pixel-wise prediction tasks such as semantic segmentation.
Ranked #26 on
Semantic Segmentation
on Cityscapes test
Dynamic Graph Message Passing Networks
We propose a dynamic graph message passing network, that significantly reduces the computational complexity compared to related works modelling a fully-connected graph.
Exploiting temporal context for 3D human pose estimation in the wild
We present a bundle-adjustment-based algorithm for recovering accurate 3D human pose and meshes from monocular videos.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Meta Learning Deep Visual Words for Fast Video Object Segmentation
We address this problem by considering the task of video object segmentation.
Weakly- and Semi-Supervised Panoptic Segmentation
We present a weakly supervised model that jointly performs both semantic- and instance-segmentation -- a particularly relevant problem given the substantial cost of obtaining pixel-perfect annotation for these tasks.
Ranked #31 on
Panoptic Segmentation
on Cityscapes val
On the Robustness of Semantic Segmentation Models to Adversarial Attacks
Deep Neural Networks (DNNs) have demonstrated exceptional performance on most recognition tasks such as image classification and segmentation.
Holistic, Instance-Level Human Parsing
We address this problem by segmenting the parts of objects at an instance-level, such that each pixel in the image is assigned a part label, as well as the identity of the object it belongs to.
Ranked #2 on
Multi-Human Parsing
on PASCAL-Part
Pixelwise Instance Segmentation with a Dynamically Instantiated Network
This subnetwork uses the initial category-level segmentation, along with cues from the output of an object detector, within an end-to-end CRF to predict instances.
Ranked #8 on
Instance Segmentation
on Cityscapes test
A Projected Gradient Descent Method for CRF Inference allowing End-To-End Training of Arbitrary Pairwise Potentials
It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label class interactions are indeed better modelled by a non-Gaussian potential.
Bottom-up Instance Segmentation using Deep Higher-Order CRFs
Traditional Scene Understanding problems such as Object Detection and Semantic Segmentation have made breakthroughs in recent years due to the adoption of deep learning.
Joint Object-Material Category Segmentation from Audio-Visual Cues
It is not always possible to recognise objects and infer material properties for a scene from visual cues alone, since objects can look visually similar whilst being made of very different materials.
Higher Order Conditional Random Fields in Deep Neural Networks
Recent deep learning approaches have incorporated CRFs into Convolutional Neural Networks (CNNs), with some even training the CRF end-to-end with the rest of the network.
Ranked #55 on
Semantic Segmentation
on PASCAL Context
SemanticPaint: A Framework for the Interactive Segmentation of 3D Scenes
We present an open-source, real-time implementation of SemanticPaint, a system for geometric reconstruction, object-class segmentation and learning of 3D scenes.
