Search Results for author:
Found 38 papers, 20 papers with code
Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping
Transformer-based models have driven significant advancements in Multimodal Large Language Models (MLLMs), yet their computational costs surge drastically when scaling resolution, training data, and model parameters.
ADAM-1: AI and Bioinformatics for Alzheimer's Detection and Microbiome-Clinical Data Integrations
The Alzheimer's Disease Analysis Model Generation 1 (ADAM) is a multi-agent large language model (LLM) framework designed to integrate and analyze multi-modal data, including microbiome profiles, clinical datasets, and external knowledge bases, to enhance the understanding and detection of Alzheimer's disease (AD).
Planarian Neural Networks: Evolutionary Patterns from Basic Bilateria Shaping Modern Artificial Neural Network Architectures
This study examined the viability of enhancing the prediction accuracy of artificial neural networks (ANNs) in image classification tasks by developing ANNs with evolution patterns similar to those of biological neural networks.
Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight
This paper introduces Chain-of-Sight, a vision-language bridge module that accelerates the pre-training of Multimodal Large Language Models (MLLMs).
Towards Better Vision-Inspired Vision-Language Models
In this paper we present a vision-inspired vision-language connection module dubbed as VIVL which efficiently exploits the vision cue for VL models.
SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery
Prior studies on Remote Sensing Foundation Model (RSFM) reveal immense potential towards a generic model for Earth Observation.
Ranked #1 on
Zero-shot Classification (unified classes)
on AID
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
When pre-training on the large-scale Kinetics-710, we achieve 89. 7% on Kinetics-400 with a frozen ViT-L model, which verifies the scalability of DiST.
Towards Real-World Visual Tracking with Temporal Contexts
To handle those problems, we propose a two-level framework (TCTrack) that can exploit temporal contexts efficiently.
Temporally-Adaptive Models for Efficient Video Understanding
Spatial convolutions are extensively used in numerous deep video models.
Ranked #4 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Rethinking Efficient Tuning Methods from a Unified Perspective
The U-Tuning framework can simultaneously encompass existing methods and derive new approaches for parameter-efficient transfer learning, which prove to achieve on-par or better performances on CIFAR-100 and FGVC datasets when compared with existing PETL methods.
Physically Plausible Animation of Human Upper Body from a Single Image
We present a new method for generating controllable, dynamically responsive, and photorealistic human animations.
Progressive Learning without Forgetting
Learning from changing tasks and sequential experience without forgetting the obtained knowledge is a challenging problem for artificial neural networks.
PVT++: A Simple End-to-End Latency-Aware Visual Tracking Framework
Visual object tracking is essential to intelligent robots.
RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection
The task of Human-Object Interaction (HOI) detection targets fine-grained visual parsing of humans interacting with their environment, enabling a broad range of applications.
Ranked #18 on
Human-Object Interaction Detection
on HICO-DET
MAR: Masked Autoencoders for Efficient Action Recognition
Inspired by this, we propose propose Masked Action Recognition (MAR), which reduces the redundant computation by discarding a proportion of patches and operating only on a part of the videos.
Ranked #11 on
Action Recognition
on Something-Something V2
Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistency
In this work, we aim to learn representations by leveraging more abundant information in untrimmed videos.
TCTrack: Temporal Contexts for Aerial Tracking
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers.
TAda! Temporally-Adaptive Convolutions for Video Understanding
This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos.
Ranked #67 on
Action Recognition
on Something-Something V2
(using extra training data)
ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning
The visualizations show that ParamCrop adaptively controls the center distance and the IoU between two augmented views, and the learned change in the disparity along the training process is beneficial to learning a strong representation.
Support-Set Based Cross-Supervision for Video Grounding
The contrastive objective aims to learn effective representations by contrastive learning, while the caption objective can train a powerful video encoder supervised by texts.
Exploring Stronger Feature for Temporal Action Localization
Temporal action localization aims to localize starting and ending time with action category.
Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling
Then our proposed Local-Global Background Modeling Network (LGBM-Net) is trained to localize instances by using only video-level labels based on Multi-Instance Learning (MIL).
Weakly-supervised Learning
Weakly-supervised Temporal Action Localization
+1
Proposal Relation Network for Temporal Action Detection
We calculate the detection results by assigning the proposals with corresponding classification results.
Ranked #3 on
Temporal Action Localization
on ActivityNet-1.3
(using extra training data)
Relation Modeling in Spatio-Temporal Action Localization
This paper presents our solution to the AVA-Kinetics Crossover Challenge of ActivityNet workshop at CVPR 2021.
Ranked #4 on
Spatio-Temporal Action Localization
on AVA-Kinetics
(using extra training data)
A Stronger Baseline for Ego-Centric Action Detection
This technical report analyzes an egocentric video action detection method we used in the 2021 EPIC-KITCHENS-100 competition hosted in CVPR2021 Workshop.
Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition
In this paper, we present empirical results for training a stronger video vision transformer on the EPIC-KITCHENS-100 Action Recognition dataset.
Multi-Scale Feature Aggregation by Cross-Scale Pixel-to-Region Relation Operation for Semantic Segmentation
Experiment results on two challenging datasets Cityscapes and COCO demonstrate that the RSP head performs competitively on both semantic segmentation and panoptic segmentation with high efficiency.
Self-supervised Motion Learning from Static Images
We furthermore introduce a static mask in pseudo motions to create local motion patterns, which forces the model to additionally locate notable motion areas for the correct classification. We demonstrate that MoSI can discover regions with large motion even without fine-tuning on the downstream datasets.
Self-Supervised Video Representation Learning with Constrained Spatiotemporal Jigsaw
With the constrained jigsaw puzzles, instead of solving them directly, which could still be extremely hard, we carefully design four surrogate tasks that are more solvable but meanwhile still ensure that the learned representation is sensitive to spatiotemporal continuity at both the local and global levels.
Toward Accurate Person-level Action Recognition in Videos of Crowded Scenes
Prior works always fail to deal with this problem in two aspects: (1) lacking utilizing information of the scenes; (2) lacking training data in the crowd and complex scenes.
A Simple Baseline for Pose Tracking in Videos of Crowded Scenes
This paper presents our solution to ACM MM challenge: Large-scale Human-centric Video Analysis in Complex Events\cite{lin2020human}; specifically, here we focus on Track3: Crowd Pose Tracking in Complex Events.
Towards Accurate Human Pose Estimation in Videos of Crowded Scenes
In this paper, we focus on improving human pose estimation in videos of crowded scenes from the perspectives of exploiting temporal context and collecting new data.
Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications
Our contributions are twofold: a) a novel dense connected prediction (DCP) layer is proposed to provide better object-level depth estimation and b) specifically for autonomous driving scenarios, dense geometrical constrains (DGC) is introduced so that precise scale factor can be recovered without additional cost for autonomous vehicles.
Ranked #63 on
Monocular Depth Estimation
on KITTI Eigen split
AutoTrack: Towards High-Performance Visual Tracking for UAV with Automatic Spatio-Temporal Regularization
Considerable tests in the indoor practical scenarios have proven the effectiveness and versatility of our localization method.
Keyfilter-Aware Real-Time UAV Object Tracking
Correlation filter-based tracking has been widely applied in unmanned aerial vehicle (UAV) with high efficiency.
Augmented Memory for Correlation Filters in Real-Time UAV Tracking
The outstanding computational efficiency of discriminative correlation filter (DCF) fades away with various complicated improvements.
Boundary Effect-Aware Visual Tracking for UAV with Online Enhanced Background Learning and Multi-Frame Consensus Verification
Meanwhile, convolutional features are extracted to provide a more comprehensive representation of the object.
Learning Aberrance Repressed Correlation Filters for Real-Time UAV Tracking
Traditional framework of discriminative correlation filters (DCF) is often subject to undesired boundary effects.
