Search Results for author:
Found 16 papers, 9 papers with code
One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos
We introduce VideoLISA, a video-based multimodal large language model designed to tackle the problem of language-instructed reasoning segmentation in videos.
VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computation
Our method, VideoLLM-MoD, is inspired by mixture-of-depths LLMs and addresses the challenge of numerous vision tokens in long-term or streaming video.
Learning Video Context as Interleaved Multimodal Sequences
In this paper, we introduce MovieSeq, a multimodal language model developed to address the wide range of challenges in understanding video contexts.
VideoLLM-online: Online Video Large Language Model for Streaming Video
Recent Large Language Models have been enhanced with vision capabilities, enabling them to comprehend images, videos, and interleaved vision-language content.
From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition
People's social relationships are often manifested through their surroundings, with certain objects or interactions acting as symbols for specific relationships, e. g., wedding rings, roses, hugs, or holding hands.
Bootstrapping SparseFormers from Vision Foundation Models
In this paper, we propose to bootstrap SparseFormers from ViT-based vision foundation models in a simple and efficient way.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
UniVTG: Towards Unified Video-Language Temporal Grounding
Most methods in this direction develop taskspecific models that are trained with type-specific labels, such as moment retrieval (time interval) and highlight detection (worthiness curve), which limits their abilities to generalize to various VTG tasks and labels.
Ranked #6 on
Natural Language Moment Retrieval
on TACoS
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn
2) Flexible inputs and intermediate results.
Affordance Grounding from Demonstration Video to Target Image
Humans excel at learning from expert demonstrations and solving their own problems.
Ranked #1 on
Video-to-image Affordance Grounding
on EPIC-Hotspot
AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistant
In this paper, we define a new task called Affordance-centric Question-driven Task Completion, where the AI assistant should learn from instructional videos to provide step-by-step help in the user's view.
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training
A standard hardware bottleneck when training deep neural networks is GPU memory.
Foreground-Background Imbalance Problem in Deep Object Detectors: A Review
Recent years have witnessed the remarkable developments made by deep learning techniques for object detection, a fundamentally challenging problem of computer vision.
Long-term Joint Scheduling for Urban Traffic
Recently, the traffic congestion in modern cities has become a growing worry for the residents.
Is Heuristic Sampling Necessary in Training Deep Object Detectors?
In this paper, we challenge the necessity of such hard/soft sampling methods for training accurate deep object detectors.
Residual Objectness for Imbalance Reduction
For a long time, object detectors have suffered from extreme imbalance between foregrounds and backgrounds.
