Search Results for author:
Found 23 papers, 12 papers with code
Learning from Streaming Video with Orthogonal Gradients
We demonstrate the drop in performance when moving from shuffled to sequential learning on three tasks: the one-video representation learning method DoRA, standard VideoMAE on multi-video datasets, and the task of future video prediction.
It's Just Another Day: Unique Video Captioning by Discriminative Prompting
We introduce two benchmarks for unique captioning, based on egocentric footage and timeloop movies - where repeating actions are common.
AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description
Our objective is to generate Audio Descriptions (ADs) for both movies and TV series in a training-free manner.
CountGD: Multi-Modal Open-World Counting
The goal of this paper is to improve the generality and accuracy of open-vocabulary object counting in images.
Ranked #1 on
Object Counting
on FSC147
AutoAD III: The Prequel -- Back to the Pixels
Generating Audio Description (AD) for movies is a challenging task that requires fine-grained visual understanding and an awareness of the characters and their names.
Stale Diffusion: Hyper-realistic 5D Movie Generation Using Old-school Methods
As the horses have long left the barn, our proposal may be seen as antiquated and irrelevant.
AutoAD III: The Prequel - Back to the Pixels
Generating Audio Description (AD) for movies is a challenging task that requires fine-grained visual understanding and an awareness of the characters and their names.
Multi-Sentence Grounding for Long-term Instructional Video
In this paper, we aim to establish an automatic, scalable pipeline for denoising the large-scale instructional dataset and construct a high-quality video-text dataset with multiple descriptive steps supervision, named HowToStep.
AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description
Audio Description (AD) is the task of generating descriptions of visual content, at suitable time intervals, for the benefit of visually impaired audiences.
Learning to Count without Annotations
While recent supervised methods for reference-based object counting continue to improve the performance on benchmark datasets, they have to rely on small datasets due to the cost associated with manually annotating dozens of objects in images.
Open-world Text-specified Object Counting
Our objective is open-world object counting in images, where the target object class is specified by a text description.
Ranked #5 on
Zero-Shot Counting
on FSC147
AutoAD: Movie Description in Context
The objective of this paper is an automatic Audio Description (AD) model that ingests movies and outputs AD in text form.
AutoAD II: The Sequel - Who, When, and What in Movie Audio Description
Audio Description (AD) is the task of generating descriptions of visual content, at suitable time intervals, for the benefit of visually impaired audiences.
Prompt Generation Networks for Input-Space Adaptation of Frozen Vision Transformers
With the introduction of the transformer architecture in computer vision, increasing model scale has been demonstrated as a clear path to achieving performance and robustness gains.
Turbo Training with Token Dropout
The objective of this paper is an efficient training method for video tasks.
Flamingo: a Visual Language Model for Few-Shot Learning
Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research.
Ranked #1 on
Action Recognition
on RareAct
Temporal Alignment Networks for Long-term Video
The objective of this paper is a temporal alignment network that ingests long term video sequences, and associated text sentences, in order to: (1) determine if a sentence is alignable with the video; and (2) if it is alignable, then determine its alignment.
Prompting Visual-Language Models for Efficient Video Understanding
Image-based visual-language (I-VL) pre-training has shown great success for learning joint visual-textual representations from large-scale web data, revealing remarkable ability for zero-shot generalisation.
Ranked #5 on
Zero-Shot Action Detection
on ActivityNet-1.3
Self-supervised Co-training for Video Representation Learning
The objective of this paper is visual-only self-supervised video representation learning.
Ranked #8 on
Self-Supervised Action Recognition Linear
on HMDB51
Memory-augmented Dense Predictive Coding for Video Representation Learning
The objective of this paper is self-supervised learning from video, in particular for representations for action recognition.
Video Representation Learning by Dense Predictive Coding
The objective of this paper is self-supervised learning of spatio-temporal embeddings from video, suitable for human action recognition.
Ranked #10 on
Self-Supervised Action Recognition Linear
on UCF101
Representation Learning
Self-Supervised Action Recognition
+3
Human Action Forecasting by Learning Task Grammars
For effective human-robot interaction, it is important that a robotic assistant can forecast the next action a human will consider in a given task.
Human Pose Forecasting via Deep Markov Models
Human pose forecasting is an important problem in computer vision with applications to human-robot interaction, visual surveillance, and autonomous driving.
