Search Results for author:
Found 32 papers, 26 papers with code
DyDiT++: Dynamic Diffusion Transformers for Efficient Visual Generation
Our investigations reveal that these costs primarily stem from the \emph{static} inference paradigm, which inevitably introduces redundant computation in certain \emph{diffusion timesteps} and \emph{spatial regions}.
Bridging the Divide: Reconsidering Softmax and Linear Attention
Widely adopted in modern Vision Transformer designs, Softmax attention can effectively capture long-range visual information; however, it incurs excessive computational cost when dealing with high-resolution inputs.
A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs
Vision-language models (VLMs) have shown remarkable success across various multi-modal tasks, yet large VLMs encounter significant efficiency challenges due to processing numerous visual tokens.
Ranked #19 on
Visual Question Answering
on MM-Vet
ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis
At the spatial level, we disentangle the computations of visible and mask tokens by encoding visible tokens independently, while decoding mask tokens conditioned on the fully encoded visible tokens.
DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution
MLLMs have demonstrated remarkable comprehension and reasoning capabilities with complex language and visual data.
Exploring contextual modeling with linear complexity for point cloud segmentation
Recently, Mamba has emerged as a promising alternative, offering efficient long-range contextual modeling capabilities without the quadratic complexity associated with Transformer's attention mechanisms.
Dynamic Diffusion Transformer
In addition, we design a Spatial-wise Dynamic Token (SDT) strategy to avoid redundant computation at unnecessary spatial locations.
Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation
Building on the observation that VLMs pre-trained on global-pooled image-text features often fail to capture fine-grained semantics necessary for effective mask classification, we propose a novel Fine-grained Semantic Adaptation (FISA) method to address this limitation.
OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength
Differentiable architecture search (DARTS) has emerged as a promising technique for effective neural architecture search, and it mainly contains two steps to find the high-performance architecture: First, the DARTS supernet that consists of mixed operations will be optimized via gradient descent.
Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
In response to this observation, we present a novel diffusion transformer framework incorporating an additional set of mediator tokens to engage with queries and keys separately.
UniTTA: Unified Benchmark and Versatile Framework Towards Realistic Test-Time Adaptation
To this end, we propose a Unified Test-Time Adaptation (UniTTA) benchmark, which is comprehensive and widely applicable.
DyFADet: Dynamic Feature Aggregation for Temporal Action Detection
Based on DFA, the proposed dynamic encoder layer aggregates the temporal features within the action time ranges and guarantees the discriminability of the extracted representations.
Ranked #3 on
Temporal Action Localization
on HACS
Demystify Mamba in Vision: A Linear Attention Perspective
By exploring the similarities and disparities between the effective Mamba and subpar linear attention Transformer, we provide comprehensive analyses to demystify the key factors behind Mamba's success.
EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
These patterns, when observed through frequency and spatial domains, incorporate lower-frequency components, and the natural image contents without distortion or data augmentation.
Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation
Existing parameter-efficient fine-tuning (PEFT) methods have achieved significant success on vision transformers (ViTs) adaptation by improving parameter efficiency.
GRA: Detecting Oriented Objects through Group-wise Rotating and Attention
GRA can adaptively capture fine-grained features of objects with diverse orientations, comprising two key components: Group-wise Rotating and Group-wise Attention.
SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning
Recent advancements in semi-supervised learning have focused on a more realistic yet challenging task: addressing imbalances in labeled data while the class distribution of unlabeled data remains both unknown and potentially mismatched.
Mask Grounding for Referring Image Segmentation
To tackle this challenge, we introduce a novel Mask Grounding auxiliary task that significantly improves visual grounding within language features, by explicitly teaching the model to learn fine-grained correspondence between masked textual tokens and their matching visual objects.
GSVA: Generalized Segmentation via Multimodal Large Language Models
Generalized Referring Expression Segmentation (GRES) extends the scope of classic RES to refer to multiple objects in one expression or identify the empty targets absent in the image.
Ranked #1 on
Generalized Referring Expression Segmentation
on gRefCOCO
(using extra training data)
Agent Attention: On the Integration of Softmax and Linear Attention
Specifically, the Agent Attention, denoted as a quadruple $(Q, A, K, V)$, introduces an additional set of agent tokens $A$ into the conventional attention module.
Fine-grained Recognition with Learnable Semantic Data Augmentation
Since images belonging to the same meta-category usually share similar visual appearances, mining discriminative visual cues is the key to distinguishing fine-grained categories.
Latency-aware Unified Dynamic Networks for Efficient Image Recognition
Dynamic computation has emerged as a promising avenue to enhance the inference efficiency of deep networks.
Computation-efficient Deep Learning for Computer Vision: A Survey
Over the past decade, deep learning models have exhibited considerable advancements, reaching or even exceeding human-level performance in a range of visual perception tasks.
FLatten Transformer: Vision Transformer using Focused Linear Attention
The quadratic computation complexity of self-attention has been a persistent challenge when applying Transformer models to vision tasks.
Dynamic Perceiver for Efficient Visual Recognition
Early exits are placed exclusively within the classification branch, thus eliminating the need for linear separability in low-level features.
Adaptive Rotated Convolution for Rotated Object Detection
In our ARC module, the convolution kernels rotate adaptively to extract object features with varying orientations in different images, and an efficient conditional computation mechanism is introduced to accommodate the large orientation variations of objects within an image.
Ranked #3 on
Oriented Object Detection
on DOTA 1.0
Latency-aware Spatial-wise Dynamic Networks
The latency prediction model can efficiently estimate the inference latency of dynamic networks by simultaneously considering algorithms, scheduling strategies, and hardware properties.
Learning to Weight Samples for Dynamic Early-exiting Networks
Intuitively, easy samples, which generally exit early in the network during inference, should contribute more to training early classifiers.
CAM-loss: Towards Learning Spatially Discriminative Feature Representations
The backbone of traditional CNN classifier is generally considered as a feature extractor, followed by a linear layer which performs the classification.
Adaptive Focus for Efficient Video Recognition
In this paper, we explore the spatial redundancy in video recognition with the aim to improve the computational efficiency.
Dynamic Neural Networks: A Survey
Dynamic neural network is an emerging research topic in deep learning.
Resolution Adaptive Networks for Efficient Inference
Adaptive inference is an effective mechanism to achieve a dynamic tradeoff between accuracy and computational cost in deep networks.
