Search Results for author:
Found 38 papers, 18 papers with code
Generative Timelines for Instructed Visual Assembly
The objective of this work is to manipulate visual timelines (e. g. a video) through natural language instructions, making complex timeline editing tasks accessible to non-expert or potentially even disabled users.
Sync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets
Temporal video alignment aims to synchronize the key events like object interactions or action phase transitions in two videos.
Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval
In this work, we consider the task of paraphrased text-to-image retrieval where a model aims to return similar results given a pair of paraphrased queries.
Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models
While there has been significant progress in customizing text-to-image generation models, generating images that combine multiple personalized concepts remains challenging.
Scaling Up Video Summarization Pretraining with Large Language Models
Long-form video content constitutes a significant portion of internet traffic, making automated video summarization an essential research problem.
Towards Automated Movie Trailer Generation
Movie trailers are an essential tool for promoting films and attracting audiences.
Long-range Multimodal Pretraining for Movie Understanding
While great progress has been attained, there is still a need for a pretrained multimodal model that can perform well in the ever-growing set of movie understanding tasks the community has been establishing.
Meta-Personalizing Vision-Language Models to Find Named Instances in Video
Large-scale vision-language models (VLM) have shown impressive results for language-guided search applications.
Localizing Moments in Long Video Via Multimodal Guidance
In this paper, we propose a method for improving the performance of natural language grounding in long videos by identifying and pruning out non-describable windows.
Ranked #3 on
Natural Language Moment Retrieval
on MAD
Natural Language Moment Retrieval
Natural Language Visual Grounding
+2
PIVOT: Prompting for Video Continual Learning
In this paper, we address the problem of continual learning for video data.
Videogenic: Identifying Highlight Moments in Videos with Professional Photographs as a Prior
This paper investigates the challenge of extracting highlight moments from videos.
VideoMap: Supporting Video Editing Exploration, Brainstorming, and Prototyping in the Latent Space
Video editing is a creative and complex endeavor and we believe that there is potential for reimagining a new video editing interface to better support the creative and exploratory nature of video editing.
The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing
Machine learning is transforming the video editing industry.
Video-ReTime: Learning Temporally Varying Speediness for Time Remapping
Furthermore, we propose an optimization for video re-timing that enables precise control over the target duration and performs more robustly on longer videos than prior methods.
FitCLIP: Refining Large-Scale Pretrained Image-Text Models for Zero-Shot Video Understanding Tasks
Large-scale pretrained image-text models have shown incredible zero-shot performance in a handful of tasks, including video ones such as action recognition and text-to-video retrieval.
OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos
We validate our approach in two large-scale datasets, EPIC-Kitchens, and HOMAGE.
vCLIMB: A Novel Video Class Incremental Learning Benchmark
We perform in-depth evaluations of existing CL methods in vCLIMB, and observe two unique challenges in video data.
MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions
The recent and increasing interest in video-language research has driven the development of large-scale datasets that enable data-intensive machine learning techniques.
Ranked #4 on
Natural Language Moment Retrieval
on MAD
MovieCuts: A New Dataset and Benchmark for Cut Type Recognition
Advances in automatic Cut-type recognition can unleash new experiences in the video editing industry, such as movie analysis for education, video re-editing, virtual cinematography, machine-assisted trailer generation, machine-assisted video editing, among others.
Learning to Cut by Watching Movies
Video content creation keeps growing at an incredible pace; yet, creating engaging stories remains challenging and requires non-trivial video editing expertise.
Transcript to Video: Efficient Clip Sequencing from Texts
To meet the demands for non-experts, we present Transcript-to-Video -- a weakly-supervised framework that uses texts as input to automatically create video sequences from an extensive collection of shots.
APES: Audiovisual Person Search in Untrimmed Video
To showcase the potential of our new dataset, we propose an audiovisual baseline and benchmark for person retrieval.
MAAS: Multi-modal Assignation for Active Speaker Detection
Active speaker detection requires a solid integration of multi-modal cues.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Real-time Semantic Segmentation with Fast Attention
The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism and captures the same rich spatial context at a small fraction of the computational cost, by changing the order of operations.
Ranked #32 on
Semantic Segmentation
on DensePASS
Active Speakers in Context
Current methods for active speak er detection focus on modeling short-term audiovisual information from a single speaker.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Temporally Distributed Networks for Fast Video Semantic Segmentation
We present TDNet, a temporally distributed network designed for fast and accurate video semantic segmentation.
Ranked #2 on
Video Semantic Segmentation
on Cityscapes val
Rethinking Online Action Detection in Untrimmed Videos: A Novel Online Evaluation Protocol
Our results confirm the problems of the previous evaluation protocols, and suggest that an IA-based protocol is more adequate to the online scenario.
The Instantaneous Accuracy: a Novel Metric for the Problem of Online Human Behaviour Recognition in Untrimmed Videos
The problem of Online Human Behaviour Recognition in untrimmed videos, aka Online Action Detection (OAD), needs to be revisited.
RefineLoc: Iterative Refinement for Weakly-Supervised Action Localization
RefineLoc shows competitive results with the state-of-the-art in weakly-supervised temporal localization.
Temporal Localization
Weakly Supervised Action Localization
+2
What do I Annotate Next? An Empirical Study of Active Learning for Action Localization
In this paper, we introduce a novel active learning framework for temporal localization that aims to mitigate this data dependency issue.
The ActivityNet Large-Scale Activity Recognition Challenge 2018 Summary
The guest tasks focused on complementary aspects of the activity recognition problem at large scale and involved three challenging and recently compiled datasets: the Kinetics-600 dataset from Google DeepMind, the AVA dataset from Berkeley and Google, and the Moments in Time dataset from MIT and IBM Research.
Diagnosing Error in Temporal Action Detectors
Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?)
ActivityNet Challenge 2017 Summary
The ActivityNet Large Scale Activity Recognition Challenge 2017 Summary: results and challenge participants papers.
SCC: Semantic Context Cascade for Efficient Action Detection
Despite the recent advances in large-scale video analysis, action detection remains as one of the most challenging unsolved problems in computer vision.
Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization
To address this need, we propose the new problem of action spotting in video, which we define as finding a specific action in a video while observing a small portion of that video.
Fast Temporal Activity Proposals for Efficient Detection of Human Actions in Untrimmed Videos
In many large-scale video analysis scenarios, one is interested in localizing and recognizing human activities that occur in short temporal intervals within long untrimmed videos.
ActivityNet: A Large-Scale Video Benchmark for Human Activity Understanding
In spite of many dataset efforts for human action recognition, current computer vision algorithms are still severely limited in terms of the variability and complexity of the actions that they can recognize.
Robust Manhattan Frame Estimation From a Single RGB-D Image
This paper proposes a new framework for estimating the Manhattan Frame (MF) of an indoor scene from a single RGB-D image.
