Search Results for author:
Found 14 papers, 11 papers with code
PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding
In this paper, we study building a Perception Language Model (PLM) in a fully open and reproducible framework for transparent research in image and video understanding.
Ranked #2 on
Video Question Answering
on TVBench
Perception Encoder: The best visual embeddings are not at the output of the network
Together with the core contrastive checkpoint, our PE family of models achieves state-of-the-art performance on a wide variety of tasks, including zero-shot image and video classification and retrieval; document, image, and video Q&A; and spatial tasks such as detection, depth estimation, and tracking.
Ranked #1 on
Object Detection
on COCO minival
(using extra training data)
Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders
We address the problem of gaze target estimation, which aims to predict where a person is looking in a scene.
Window Attention is Bugged: How not to Interpolate Position Embeddings
To fix it, we introduce a simple absolute window position embedding strategy, which solves the bug outright in Hiera and allows us to increase both speed and performance of the model in ViTDet.
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
Modern hierarchical vision transformers have added several vision-specific components in the pursuit of supervised classification performance.
Ranked #1 on
Image Classification
on iNaturalist 2019
(using extra training data)
ZipIt! Merging Models from Different Tasks without Training
While this works for models trained on the same task, we find that this fails to account for the differences in models trained on disjoint tasks.
Token Merging for Fast Stable Diffusion
In the process, we speed up image generation by up to 2x and reduce memory consumption by up to 5. 6x.
Token Merging: Your ViT But Faster
Off-the-shelf, ToMe can 2x the throughput of state-of-the-art ViT-L @ 512 and ViT-H @ 518 models on images and 2. 2x the throughput of ViT-L on video with only a 0. 2-0. 3% accuracy drop in each case.
Ranked #13 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Hydra Attention: Efficient Attention with Many Heads
While transformers have begun to dominate many tasks in vision, applying them to large images is still computationally difficult.
Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
Convolutional Neural Networks have been shown to be vulnerable to adversarial examples, which are known to locate in subspaces close to where normal data lies but are not naturally occurring and of low probability.
TIDE: A General Toolbox for Identifying Object Detection Errors
We introduce TIDE, a framework and associated toolbox for analyzing the sources of error in object detection and instance segmentation algorithms.
YOLACT++: Better Real-time Instance Segmentation
Then we produce instance masks by linearly combining the prototypes with the mask coefficients.
Ranked #14 on
Real-time Instance Segmentation
on MSCOCO
YOLACT: Real-time Instance Segmentation
Then we produce instance masks by linearly combining the prototypes with the mask coefficients.
Ranked #20 on
Real-time Instance Segmentation
on MSCOCO
