Search Results for author:
Found 14 papers, 13 papers with code
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding
However, existing LLM-based large multimodal models (e. g., Video-LLaMA, VideoChat) can only take in a limited number of frames for short video understanding.
Ranked #1 on
Video Classification
on COIN
VideoGLUE: Video General Understanding Evaluation of Foundation Models
We evaluate existing foundation models video understanding capabilities using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition, temporal localization, and spatiotemporal localization), eight datasets well received by the community, and four adaptation methods tailoring a foundation model (FM) for a downstream task.
Emergent Correspondence from Image Diffusion
We propose a simple strategy to extract this implicit knowledge out of diffusion networks as image features, namely DIffusion FeaTures (DIFT), and use them to establish correspondences between real images.
PromptFusion: Decoupling Stability and Plasticity for Continual Learning
Current research on continual learning mainly focuses on relieving catastrophic forgetting, and most of their success is at the cost of limiting the performance of newly incoming tasks.
Searching for Structure in Unfalsifiable Claims
In this work, we aim to distill such posts into a small set of narratives that capture the essential claims related to a given topic.
Visual Prompt Tuning
The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, ie, full fine-tuning.
Ranked #2 on
Prompt Engineering
on ImageNet-21k
Rethinking Nearest Neighbors for Visual Classification
In this paper, we investigate $k$-Nearest-Neighbor (k-NN) classifiers, a classical model-free learning method from the pre-deep learning era, as an augmentation to modern neural network based approaches.
When in Doubt: Improving Classification Performance with Alternating Normalization
We introduce Classification with Alternating Normalization (CAN), a non-parametric post-processing step for classification.
Exploring Visual Engagement Signals for Representation Learning
Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes.
Intentonomy: a Dataset and Study towards Human Intent Understanding
Based on our findings, we conduct further study to quantify the effect of attending to object and context classes as well as textual information in the form of hashtags when training an intent classifier.
Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset
In this work we explore the task of instance segmentation with attribute localization, which unifies instance segmentation (detect and segment each object instance) and fine-grained visual attribute categorization (recognize one or multiple attributes).
Deep Multi-Modal Sets
Most deep learning-based methods rely on a late fusion technique whereby multiple feature types are encoded and concatenated and then a multi layer perceptron (MLP) combines the fused embedding to make predictions.
Class-Balanced Loss Based on Effective Number of Samples
We design a re-weighting scheme that uses the effective number of samples for each class to re-balance the loss, thereby yielding a class-balanced loss.
Ranked #2 on
Long-tail Learning
on EGTEA
A Deep-Learning-Based Fashion Attributes Detection Model
Such information analyzing process is called abstracting, which recognize similarities or differences across all the garments and collections.
