Search Results for author:
Found 56 papers, 26 papers with code
MPII: Multi-Level Mutual Promotion for Inference and Interpretation
In this paper, we propose a multi-level Mutual Promotion mechanism for self-evolved Inference and sentence-level Interpretation (MPII).
ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning
Video understanding is inherently intention-driven-humans naturally focus on relevant frames based on their goals.
JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers
We present JointDiT, a diffusion transformer that models the joint distribution of RGB and depth.
Subject-driven Video Generation via Disentangled Identity and Motion
We propose to train a subject-driven customized video generation model through decoupling the subject-specific learning from temporal dynamics in zero-shot without additional tuning.
Securing the Skies: A Comprehensive Survey on Anti-UAV Methods, Benchmarking, and Future Directions
Unmanned Aerial Vehicles (UAVs) are indispensable for infrastructure inspection, surveillance, and related tasks, yet they also introduce critical security challenges.
MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance
However, existing methods struggle with complex object movements and multi-object motion control, resulting in imprecise trajectory adherence, poor object consistency, and compromised visual quality.
HA-VLN: A Benchmark for Human-Aware Navigation in Discrete-Continuous Environments with Dynamic Multi-Human Interactions, Real-World Validation, and an Open Leaderboard
Vision-and-Language Navigation (VLN) systems often focus on either discrete (panoramic) or continuous (free-motion) paradigms alone, overlooking the complexities of human-populated, dynamic environments.
HiTVideo: Hierarchical Tokenizers for Enhancing Text-to-Video Generation with Autoregressive Large Language Models
We explore the trade-offs between compression and reconstruction, while emphasizing the advantages of high-compressed semantic tokens in text-to-video tasks.
A Demo of Radar Sensing Aided Rotatable Antenna for Wireless Communication System
Rotatable antenna (RA) represents a novel antenna architecture that enhances wireless communication system performance by independently or collectively adjusting each antenna's boresight/orientation.
FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesis
To synthesize natural object motion while supporting detailed camera control, our framework adopts a two-stage video synthesis pipeline consisting of optical flow generation and flow-conditioned video synthesis.
HomoGen: Enhanced Video Inpainting via Homography Propagation and Diffusion
In this paper, we present HomoGen, an enhanced video inpainting method based on homography propagation and diffusion models.
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Second, Target Prompt Generation creates dynamic prompts by attending to masked source prompts, enabling seamless adaptation to unseen domains and classes.
MageBench: Bridging Large Multimodal Models to Agents
Such vision-in-the-chain reasoning paradigm is more aligned with the needs of multimodal agents, while being rarely evaluated.
StableAnimator: High-Quality Identity-Preserving Human Image Animation
During inference, we propose a novel Hamilton-Jacobi-Bellman (HJB) equation-based optimization to further enhance the face quality.
REDUCIO! Generating 1024$\times$1024 Video within 16 Seconds using Extremely Compressed Motion Latents
Commercial video generation models have exhibited realistic, high-fidelity results but are still restricted to limited access.
LLM2CLIP: Powerful Language Model Unlocks Richer Visual Representation
CLIP is a foundational multimodal model that aligns image and text features into a shared representation space via contrastive learning on large-scale image-text pairs.
Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions
Vision-and-Language Navigation (VLN) aims to develop embodied agents that navigate based on human instructions.
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms
In this paper, we target the realm of visual aesthetics and aim to align vision models with human aesthetic standards in a retrieval system.
AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction
Text-guided video prediction (TVP) involves predicting the motion of future frames from the initial frame according to an instruction, which has wide applications in virtual reality, robotics, and content creation.
MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion
In this paper, we propose MotionFollower, a lightweight score-guided diffusion model for video motion editing.
Effectiveness of Self-Assessment Software to Evaluate Preclinical Operative Procedures
Class II preparation at 400{\mu}m tolerance had the smallest mean difference of 0. 41 points.
An edge detection-based deep learning approach for tear meniscus height measurement
For improved segmentation of the pupil and tear meniscus areas, the convolutional neural network Inceptionv3 was first implemented as an image quality assessment model, effectively identifying higher-quality images with an accuracy of 98. 224%.
BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition
To remedy this we propose a two-fold strategy: (1) We introduce an innovative approach that encodes bone connectivity by harnessing the power of graph distances to describe the physical topology; we further incorporate action-specific topological representation via persistent homology analysis to depict systemic dynamics.
Ranked #6 on
Skeleton Based Action Recognition
on NTU RGB+D 120
MotionEditor: Editing Video Motion via Content-Aware Diffusion
This mechanism enables the editing branch to query the key and value from the reconstruction branch in a decoupled manner, making the editing branch retain the original background and protagonist appearance.
VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Models
Our model can edit and translate the desired results within seconds based on user instructions.
ART$\boldsymbol{\cdot}$V: Auto-Regressive Text-to-Video Generation with Diffusion Models
Second, it preserves the high-fidelity generation ability of the pre-trained image diffusion models by making only minimal network modifications.
A Survey on Video Diffusion Models
However, existing surveys mainly focus on diffusion models in the context of image generation, with few up-to-date reviews on their application in the video domain.
SimDA: Simple Diffusion Adapter for Efficient Video Generation
In this work, we propose a Simple Diffusion Adapter (SimDA) that fine-tunes only 24M out of 1. 1B parameters of a strong T2I model, adapting it to video generation in a parameter-efficient way.
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
While modeling temporal information within straight through tube is widely adopted in literature, we find that simple frame alignment already provides enough essence without temporal attention.
Ranked #21 on
Action Classification
on Kinetics-400
ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules
We evaluate ChartReader on Chart-to-Table, ChartQA, and Chart-to-Text tasks, demonstrating its superiority over existing methods.
Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarios
However, current works pursuing sentence-level explanations rely heavily on annotated training data, which limits the development of interpretability to only a few tasks.
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
With these new techniques and other designs, we show that the proposed general-purpose task-solver can perform both instance segmentation and depth estimation well.
Ranked #27 on
Monocular Depth Estimation
on NYU-Depth V2
ResFormer: Scaling ViTs with Multi-Resolution Training
We introduce, ResFormer, a framework that is built upon the seminal idea of multi-resolution training for improved performance on a wide spectrum of, mostly unseen, testing resolutions.
SVFormer: Semi-supervised Video Transformer for Action Recognition
In this paper, we investigate the use of transformer models under the SSL setting for action recognition.
GSRFormer: Grounded Situation Recognition Transformer with Alternate Semantic Attention Refinement
In the second stage, we exploit transformer layers to unearth the potential semantic relations within both verbs and semantic roles.
Rethinking Spatial Invariance of Convolutional Networks for Object Counting
We evaluate our methods on 4 mainstream object counting networks (i. e., MCNN, CSRNet, SANet, and ResNet-50).
Ranked #1 on
Object Counting
on TRANCOS
On Data Scaling in Masked Image Modeling
Our study reveals that: (i) Masked image modeling is also demanding on larger data.
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling
A key idea of efficient implementation is to discard the masked image patches (or tokens) throughout the target network (encoder), which requires the encoder to be a plain vision transformer (e. g., ViT), albeit hierarchical vision transformers (e. g., Swin Transformer) have potentially better properties in formulating vision inputs.
Deeper Insights into the Robustness of ViTs towards Common Corruptions
With Vision Transformers (ViTs) making great advances in a variety of computer vision tasks, recent literature have proposed various variants of vanilla ViTs to achieve better efficiency and efficacy.
Multi-granularity Relabeled Under-sampling Algorithm for Imbalanced Data
The Tomek-Link sampling algorithm can effectively reduce the class overlap on data, remove the majority instances that are difficult to distinguish, and improve the algorithm classification accuracy.
SimMIM: A Simple Framework for Masked Image Modeling
We also leverage this approach to facilitate the training of a 3B model (SwinV2-G), that by $40\times$ less data than that in previous practice, we achieve the state-of-the-art on four representative vision benchmarks.
Representation Learning
Self-Supervised Image Classification
+1
Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning
Cross-modal correlation provides an inherent supervision for video unsupervised representation learning.
On the Connection between Local Attention and Dynamic Depth-wise Convolution
Sparse connectivity: there is no connection across channels, and each position is connected to the positions within a small local window.
Self-Supervised Learning with Swin Transformers
We are witnessing a modeling shift from CNN to Transformers in computer vision.
Ranked #78 on
Self-Supervised Image Classification
on ImageNet
Calibration of Human Driving Behavior and Preference Using Naturalistic Traffic Data
Towards this end it is necessary that we have a comprehensive modeling framework for decision-making within which human driving preferences can be inferred statistically from observed driving behaviors in realistic and naturalistic traffic settings.
Learning to Estimate Kernel Scale and Orientation of Defocus Blur with Asymmetric Coded Aperture
Consistent in-focus input imagery is an essential precondition for machine vision systems to perceive the dynamic environment.
Temporal Action Detection with Multi-level Supervision
We extensively benchmark against the baselines for SSAD and OSAD on our created data splits in THUMOS14 and ActivityNet1. 2, and demonstrate the effectiveness of the proposed UFA and IB methods.
Towards a Systematic Computational Framework for Modeling Multi-Agent Decision-Making at Micro Level for Smart Vehicles in a Smart World
To demonstrate how our approach can be applied to realistic traffic settings, we conduct a simulation experiment: to derive merging and yielding behaviors on a double-lane highway with an unexpected barrier.
Informative Dropout for Robust Representation Learning: A Shape-bias Perspective
Specifically, we discriminate texture from shape based on local self-information in an image, and adopt a Dropout-like algorithm to decorrelate the model output from the local texture.
Reinforcing Short-Length Hashing
Due to the compelling efficiency in retrieval and storage, similarity-preserving hashing has been widely applied to approximate nearest neighbor search in large-scale image retrieval.
Weakly-Supervised Action Localization by Generative Attention Modeling
By maximizing the conditional probability with respect to the attention, the action and non-action frames are well separated.
Ranked #11 on
Weakly Supervised Action Localization
on ActivityNet-1.2
Self-supervised Object Motion and Depth Estimation from Video
We present a self-supervised learning framework to estimate the individual object motion and monocular depth from video.
Improving the Learning of Multi-column Convolutional Neural Network for Crowd Counting
By minimizing the mutual information, each column is guided to learn features with different image scales.
Learning Spatial Awareness to Improve Crowd Counting
Although the Maximum Excess over SubArrays (MESA) loss has been previously proposed to address the above issues by finding the rectangular subregion whose predicted density map has the maximum difference from the ground truth, it cannot be solved by gradient descent, thus can hardly be integrated into the deep learning framework.
Ranked #5 on
Crowd Counting
on WorldExpo’10
Decoupling Localization and Classification in Single Shot Temporal Action Detection
Each branch produces a set of action anchor layers by applying deconvolution to the feature maps of the main stream.
Ranked #28 on
Temporal Action Localization
on THUMOS’14
Recurrent Tubelet Proposal and Recognition Networks for Action Detection
The RTP initializes action proposals of the start frame through a Region Proposal Network and then estimates the movements of proposals in next frame in a recurrent manner.
