Search Results for author:
Found 42 papers, 19 papers with code
Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition
In addition, we propose a new prototypical diversity amplification loss to strengthen the model's capacity by amplifying the differences between different prototypes.
Ranked #1 on
Micro-Action Recognition
on MA-52
EnergyMoGen: Compositional Human Motion Generation with Energy-Based Diffusion Model in Latent Space
To address this issue, we propose EnergyMoGen, which includes two spectrums of Energy-Based Models: (1) We interpret the diffusion model as a latent-aware energy-based model that generates motions by composing a set of diffusion models in latent space; (2) We introduce a semantic-aware energy model based on cross-attention, which enables semantic composition and adaptive gradient descent for text embeddings.
InfiniDreamer: Arbitrarily Long Human Motion Generation via Segment Score Distillation
InfiniDreamer addresses the limitations of current motion generation methods, which are typically restricted to short sequences due to the lack of long motion training data.
CktGen: Specification-Conditioned Analog Circuit Generation
To address this limitation, we introduce a task that directly generates analog circuits based on specified specifications, termed specification-conditioned analog circuit generation.
ZeroMamba: Exploring Visual State Space Model for Zero-Shot Learning
Due to the limited receptive fields of CNNs and the quadratic complexity of ViTs, however, these visual backbones achieve suboptimal visual-semantic interactions.
Prototype Learning for Micro-gesture Classification
In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the track of Micro-gesture Classification in the MiGA challenge at IJCAI 2024.
VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation
In this paper, SDS is decoupled into a weighted sum of two components: the reconstruction term and the classifier-free guidance term.
Prompt-Aware Adapter: Towards Learning Adaptive Visual Tokens for Multimodal Large Language Models
To bridge the gap between vision and language modalities, Multimodal Large Language Models (MLLMs) usually learn an adapter that converts visual inputs to understandable tokens for Large Language Models (LLMs).
TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment
Recent advancements in image understanding have benefited from the extensive use of web image-text pairs.
Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly
In pursuit of these answers, we present a comprehensive benchmark for Causation Understanding of Video Anomaly (CUVA).
Clustering for Protein Representation Learning
We select the highest-scoring clusters and use their medoid nodes for the next iteration of clustering, until we obtain a hierarchical and informative representation of the protein.
EVA: Zero-shot Accurate Attributes and Multi-Object Video Editing
We find that the crux of the issue stems from the imprecise distribution of attention weights across designated regions, including inaccurate text-to-attribute control and attention leakage.
ProtChatGPT: Towards Understanding Proteins with Large Language Models
The protein first undergoes protein encoders and PLP-former to produce protein embeddings, which are then projected by the adapter to conform with the LLM.
HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting
Extensive experiments demonstrate the efficacy of HeadStudio in generating animatable avatars from textual prompts, exhibiting appealing appearances.
Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling
To address this problem, we propose a data-driven method for coarse motion refinement.
Uncovering What Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly
In addition we also introduce MMEval a novel evaluation metric designed to better align with human preferences for CUVA facilitating the measurement of existing LLMs in comprehending the underlying cause and corresponding effect of video anomalies.
DocMSU: A Comprehensive Benchmark for Document-level Multimodal Sarcasm Understanding
Multimodal Sarcasm Understanding (MSU) has a wide range of applications in the news field such as public opinion analysis and forgery detection.
Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigation
To this end, an agent needs to 1) learn a piece of certain knowledge about the relations of object categories in the world during training and 2) look for the target object based on the pre-learned object category relations and its moving trajectory in the current unseen environment.
A Reliable Representation with Bidirectional Transition Model for Visual Reinforcement Learning Generalization
Visual reinforcement learning has proven effective in solving control tasks with high-dimensional observations.
FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene Syntax
Text-to-video (T2V) generation is a rapidly growing research area that aims to translate the scenes, objects, and actions within complex video text into a sequence of coherent visual frames.
Prior-Free Continual Learning with Unlabeled Data in the Wild
Existing CL methods usually reduce forgetting with task priors, \ie using task identity or a subset of previously seen samples for model training.
Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos
Instead of contrasting the representations of clips or frames, in this paper, we propose a unified self-supervised framework by conducting contrastive learning at the point level.
Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos
In this paper, we propose a Masked Spatio-Temporal Structure Prediction (MaST-Pre) method to capture the structure of point cloud videos without human annotations.
DPMix: Mixture of Depth and Point Cloud Video Experts for 4D Action Segmentation
The proposed method, named Mixture of Depth and Point cloud video experts (DPMix), achieved the first place in the 4D Action Segmentation Track of the HOI4D Challenge 2023.
Keyword-Aware Relative Spatio-Temporal Graph Networks for Video Question Answering
The main challenge in video question answering (VideoQA) is to capture and understand the complex spatial and temporal relations between objects based on given questions.
A Study on Differentiable Logic and LLMs for EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2023
Our research focuses on the innovative application of a differentiable logic loss in the training to leverage the co-occurrence relations between verb and noun, as well as the pre-trained Large Language Models (LLMs) to generate the logic rules for the adaptation to unseen action labels.
Continual Learning with Strong Experience Replay
Experience Replay (ER) is a simple and effective rehearsal-based strategy, which optimizes the model with current training data and a subset of old samples stored in a memory buffer.
Text to Point Cloud Localization with Relation-Enhanced Transformer
To overcome these two challenges, we propose a unified Relation-Enhanced Transformer (RET) to improve representation discriminability for both point cloud and natural language queries.
STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i. e., sparsification and anonymization, to remove privacy from a spatio-temporal perspective.
PointListNet: Deep Learning on 3D Point Lists
Deep neural networks on regular 1D lists (e. g., natural languages) and irregular 3D sets (e. g., point clouds) have made tremendous achievements.
Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?
Vision Transformers (ViTs) have proven to be effective, in solving 2D image understanding tasks by training over large-scale image datasets; and meanwhile as a somehow separate track, in modeling the 3D visual world too such as voxels or point clouds.
SEFormer: Structure Embedding Transformer for 3D Object Detection
Effectively preserving and encoding structure features from objects in irregular and sparse LiDAR points is a key challenge to 3D object detection on point cloud.
PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
Then, a spatial convolution is employed to capture the local structure of points in the 3D space, and a temporal convolution is used to model the dynamics of the spatial regions along the time dimension.
Ranked #3 on
3D Action Recognition
on NTU RGB+D
Self-Supervised Global-Local Structure Modeling for Point Cloud Domain Adaptation With Reliable Voted Pseudo Labels
In this paper, we propose an unsupervised domain adaptation method for deep point cloud representation learning.
Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos
To capture the dynamics in point cloud videos, point tracking is usually employed.
Ranked #4 on
3D Action Recognition
on NTU RGB+D
PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing
We apply PointRNN, PointGRU and PointLSTM to moving point cloud prediction, which aims to predict the future trajectories of points in a set given their history movements.
Cascaded Revision Network for Novel Object Captioning
By this novel cascaded captioning-revising mechanism, CRN can accurately describe images with unseen objects.
Attract or Distract: Exploit the Margin of Open Set
In this paper, we exploit the semantic structure of open set data from two aspects: 1) Semantic Categorical Alignment, which aims to achieve good separability of target known classes by categorically aligning the centroid of target with the source.
Adaptive Exploration for Unsupervised Person Re-Identification
However, a problem of the adaptive selection is that, when an image has too many neighborhoods, it is more likely to attract other images as its neighborhoods.
Cubic LSTMs for Video Prediction
Predicting future frames in videos has become a promising direction of research for both computer vision and robot learning communities.
Complex Event Detection by Identifying Reliable Shots From Untrimmed Videos
relevant) to the given event class, we formulate this task as a multi-instance learning (MIL) problem by taking each video as a bag and the video shots in each video as instances.
Unsupervised Person Re-identification: Clustering and Fine-tuning
Progressively, pedestrian clustering and the CNN model are improved simultaneously until algorithm convergence.
Ranked #12 on
Unsupervised Person Re-Identification
on DukeMTMC-reID
