Search Results for author:
Found 17 papers, 7 papers with code
Training-Free Exponential Extension of Sliding Window Context with Cascading KV Cache
The context window within a transformer provides a form of active memory for the current task, which can be useful for few-shot learning and conditional generation, both which depend heavily on previous context tokens.
HiP Attention: Sparse Sub-Quadratic Attention with Hierarchical Attention Pruning
In modern large language models (LLMs), increasing sequence lengths is a crucial challenge for enhancing their comprehension and coherence in handling complex tasks such as multi-modal question answering.
PC-LoRA: Low-Rank Adaptation for Progressive Model Compression with Knowledge Distillation
Low-rank adaption (LoRA) is a prominent method that adds a small number of learnable parameters to the frozen pre-trained weights for parameter-efficient fine-tuning.
Visualizing the loss landscape of Self-supervised Vision Transformer
To further investigate the reason for better generalization of the self-supervised ViT when trained by MAE (MAE-ViT) and the effect of the gradient correction of RC-MAE from the perspective of optimization, we visualize the loss landscapes of the self-supervised vision transformer by both MAE and RC-MAE and compare them with the supervised ViT (Sup-ViT).
Dynamic and Super-Personalized Media Ecosystem Driven by Generative AI: Unpredictable Plays Never Repeating The Same
This paper introduces a media service model that exploits artificial intelligence (AI) video generators at the receive end.
KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis
As text-to-image (T2I) synthesis models increase in size, they demand higher inference costs due to the need for more expensive GPUs with larger memory, which makes it challenging to reproduce these models in addition to the restricted access to training datasets.
EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens
Masked Video Autoencoder (MVA) approaches have demonstrated their potential by significantly outperforming previous video representation learning methods.
Ranked #1 on
Object State Change Classification
on Ego4D
Object State Change Classification
Object State Change Classification on Ego4D
+4
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Masked image modeling (MIM) has become a popular strategy for self-supervised learning~(SSL) of visual representations with Vision Transformers.
MPViT: Multi-Path Vision Transformer for Dense Prediction
While Convolutional Neural Networks (CNNs) have been the dominant architectures for such tasks, recently introduced Vision Transformers (ViTs) aim to replace them as a backbone.
Ranked #38 on
Instance Segmentation
on COCO minival
Diverse Temporal Aggregation and Depthwise Spatiotemporal Factorization for Efficient Video Classification
By using the proposed temporal modeling method (T-OSA), and the efficient factorized component (D(2+1)D), we construct two types of VoV3D networks, VoV3D-M and VoV3D-L.
Ranked #30 on
Action Recognition
on Something-Something V1
(using extra training data)
Adversarial Training with Stochastic Weight Average
Moreover, we further improved SWA to be adequate to adversarial training.
Localization Uncertainty Estimation for Anchor-Free Object Detection
To tackle these limitations, we propose a new localization uncertainty estimation method called UAD for anchor-free object detection.
Ranked #128 on
Object Detection
on COCO test-dev
CenterMask: Real-Time Anchor-Free Instance Segmentation
We propose a simple yet efficient anchor-free instance segmentation, called CenterMask, that adds a novel spatial attention-guided mask (SAG-Mask) branch to anchor-free one stage object detector (FCOS) in the same vein with Mask R-CNN.
CenterMask : Real-Time Anchor-Free Instance Segmentation
We hope that CenterMask and VoVNetV2 can serve as a solid baseline of real-time instance segmentation and backbone network for various vision tasks, respectively.
Ranked #1 on
Object Detection
on COCO test-dev
(Hardware Burden metric)
An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection
As DenseNet conserves intermediate features with diverse receptive fields by aggregating them with dense connection, it shows good performance on the object detection task.
Ranked #64 on
Instance Segmentation
on COCO test-dev
Rank of Experts: Detection Network Ensemble
The recent advances of convolutional detectors show impressive performance improvement for large scale object detection.
Wide-Residual-Inception Networks for Real-time Object Detection
Since convolutional neural network(CNN)models emerged, several tasks in computer vision have actively deployed CNN models for feature extraction.
