Search Results for author:
Found 22 papers, 10 papers with code
Unleashing Hour-Scale Video Training for Long Video-Language Understanding
To close this gap, we present VideoMarathon, a large-scale hour-long video instruction-following dataset.
MOVi: Training-free Text-conditioned Multi-Object Video Generation
Recent advances in diffusion-based text-to-video (T2V) models have demonstrated remarkable progress, but these models still face challenges in generating videos with multiple objects.
Self-Taught Agentic Long Context Understanding
Answering complex, long-context questions remains a major challenge for large language models (LLMs) as it requires effective question clarifications and context retrieval.
Agent Laboratory: Using LLM Agents as Research Assistants
Historically, scientific discovery has been a lengthy and costly process, demanding substantial time and resources from initial conception to final results.
SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer
With its fully-differentiable design and semantic-rich latent space, our experiment demonstrates that SoftVQ-VAE achieves efficient tokenization without compromising generation quality, paving the way for more efficient generative models.
Koala: Key frame-conditioned long video-LLM
Long video question answering is a challenging task that involves recognizing short-term activities and reasoning about their fine-grained relationships.
CLAMP: Contrastive LAnguage Model Prompt-tuning
Our results show that LLMs can, indeed, achieve good image classification performance when adapted this way.
Label Budget Allocation in Multi-Task Learning
How should the label budget (i. e. the amount of money spent on labeling) be allocated among different tasks to achieve optimal multi-task performance?
DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations
Previous works have focused on learning the alignment between textual and visual spaces to compensate for limited image labels, yet may suffer from reduced accuracy due to the scarcity of high-quality multi-label annotations.
DIME-FM: DIstilling Multimodal and Efficient Foundation Models
We transfer the knowledge from the pre-trained CLIP-ViTL/14 model to a ViT-B/32 model, with only 40M public images and 28. 4M unpaired public sentences.
DIME-FM : DIstilling Multimodal and Efficient Foundation Models
In this paper, we introduce a new distillation mechanism (DIME-FM) that allows us to transfer the knowledge contained in large VLFMs to smaller, customized foundation models using a relatively small amount of inexpensive, unpaired images and sentences.
DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations
Solving multi-label recognition (MLR) for images in the low-label regime is a challenging task with many real-world applications.
Dynamic Network Quantization for Efficient Video Inference
Deep convolutional networks have recently achieved great success in video recognition, yet their practical realization remains a challenge due to the large amount of computational resources required to achieve robust recognition.
AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition
Specifically, given a video segment, a multi-modal policy network is used to decide what modalities should be used for processing by the recognition model, with the goal of improving both accuracy and efficiency.
Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
Second, to effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network.
Revisiting Few-shot Activity Detection with Class Similarity Control
In this paper, we present a conceptually simple and general yet novel framework for few-shot temporal activity detection based on proposal regression which detects the start and end time of the activities in untrimmed videos.
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning
Multi-task learning is an open and challenging problem in computer vision.
Ranked #118 on
Semantic Segmentation
on NYU Depth v2
Weakly-supervised Compositional FeatureAggregation for Few-shot Recognition
Learning from a few examples is a challenging task for machine learning.
Domain Agnostic Learning with Disentangled Representations
Unsupervised model transfer has the potential to greatly improve the generalizability of deep models to novel domains.
Ranked #4 on
Multi-target Domain Adaptation
on DomainNet
Similarity R-C3D for Few-shot Temporal Activity Detection
In this paper, we present a conceptually simple and general yet novel framework for few-shot temporal activity detection which detects the start and end time of the few-shot input activities in an untrimmed video.
TwoStreamVAN: Improving Motion Modeling in Video Generation
Video generation is an inherently challenging task, as it requires modeling realistic temporal dynamics as well as spatial content.
A Temporally-Aware Interpolation Network for Video Frame Inpainting
We propose the first deep learning solution to video frame inpainting, a challenging instance of the general video inpainting problem with applications in video editing, manipulation, and forensics.
