Search Results for author:
Found 46 papers, 23 papers with code
BioMoDiffuse: Physics-Guided Biomechanical Diffusion for Controllable and Authentic Human Motion Synthesis
Human motion generation holds significant promise in fields such as animation, film production, and robotics.
OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation
Comprehensive experiments demonstrate that OmniPhysGS achieves more general and realistic physical dynamics across a broader spectrum of materials, including elastic, viscoelastic, plastic, and fluid substances, as well as interactions between different materials.
DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation
Recent advancements in 3D content generation from text or a single image struggle with limited high-quality 3D datasets and inconsistency from 2D multi-view generation.
Pyramidal Flow Matching for Efficient Video Generative Modeling
Video generation requires modeling a vast spatiotemporal space, which demands significant computational resources and data usage.
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling
In the endeavor to make autonomous robots take actions, task planning is a major challenge that requires translating high-level task descriptions into long-horizon action sequences.
Local Occupancy-Enhanced Object Grasping with Multiple Triplanar Projection
This paper addresses the challenge of robotic grasping of general objects.
InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis with Semantic Graph Prior
The proposed semantic graph prior learns layout appearances and object distributions simultaneously, demonstrating versatility across various downstream tasks in a zero-shot manner.
HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors
Despite recent advancements in high-fidelity human reconstruction techniques, the requirements for densely captured images or time-consuming per-instance optimization significantly hinder their applications in broader scenarios.
Learning Solution-Aware Transformers for Efficiently Solving Quadratic Assignment Problem
Recently various optimization problems, such as Mixed Integer Linear Programming Problems (MILPs), have undergone comprehensive investigation, leveraging the capabilities of machine learning.
RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance
Our study shows that based on a recent rectified flow framework, the major limitation of vanilla classifier guidance in requiring a special classifier can be resolved with a simple fixed-point solution, allowing flexible personalization with off-the-shelf image discriminators.
Neural Assembler: Learning to Generate Fine-Grained Robotic Assembly Instructions from Multi-View Images
To tackle this, we propose an end-to-end model known as the Neural Assembler.
Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion
We address these challenges by proposing Text-controlled Motion Mamba (TM-Mamba), a unified model that integrates temporal global context, language query control, and spatial graph topology with only linear memory cost.
InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior
We introduce InstructScene, a novel generative framework that integrates a semantic graph prior and a layout decoder to improve controllability and fidelity for 3D scene synthesis.
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
In light of recent advances in multimodal Large Language Models (LLMs), there is increasing attention to scaling them from image-text data to more informative real-world videos.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Exploring Orthogonality in Open World Object Detection
To address this problem we exploit three levels of orthogonality in the detection process: First the objectness and classification heads are disentangled by operating on separate sets of features that are orthogonal to each other in a devised polar coordinate system.
Ink Dot-Oriented Differentiable Optimization for Neural Image Halftoning
In this paper we propose an innovative halftoning method termed "neural dot-controllable halftoning".
Countering Personalized Text-to-Image Generation with Influence Watermarks
In this way the personal semantics of these images are under protection even if these images are modified to some extent.
Co-Salient Object Detection with Semantic-Level Consensus Extraction and Dispersion
Given a group of images, co-salient object detection (CoSOD) aims to highlight the common salient object in each image.
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
Specifically, we introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language that LLM can read.
Regularizing Second-Order Influences for Continual Learning
Continual learning aims to learn on non-stationary data streams without catastrophically forgetting previous knowledge.
Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce
Extensive experimental results show that, without further fine-tuning, ECLIP surpasses existing methods by a large margin on a broad range of downstream tasks, demonstrating the strong transferability to real-world E-commerce applications.
Neural Koopman Pooling: Control-Inspired Temporal Dynamics Encoding for Skeleton-Based Action Recognition
Besides, we also show that our Koopman pooling framework can be easily extended to one-shot action recognition when combined with Dynamic Mode Decomposition.
Video Action Segmentation via Contextually Refined Temporal Keypoints
Video action segmentation refers to the task of densely casting each video frame or short segment in an untrimmed video into some pre-specified action categories.
Image Completion with Heterogeneously Filtered Spectral Hints
We also prove the effectiveness of our design via ablation studies, from which one may notice that the aforementioned challenges, i. e. pattern unawareness, blurry textures, and structure distortion, can be noticeably resolved.
Ranked #2 on
Image Inpainting
on FFHQ 512 x 512
Patch-based Knowledge Distillation for Lifelong Person Re-Identification
The task of lifelong person re-identification aims to match a person across multiple cameras given continuous data streams.
Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Grounding
Spatio-Temporal video grounding (STVG) focuses on retrieving the spatio-temporal tube of a specific object depicted by a free-form textual expression.
Learning Sample Importance for Cross-Scenario Video Temporal Grounding
To this end, we propose a novel method called Debiased Temporal Language Localizer (DebiasTLL) to prevent the model from naively memorizing the biases and enforce it to ground the query sentence based on true inter-modal relationship.
Complex Video Action Reasoning via Learnable Markov Logic Network
The main contributions of this work are two-fold: 1) Different from existing black-box models, the proposed model simultaneously implements the localization of temporal boundaries and the recognition of action categories by grounding the logical rules of MLN in videos.
Joint Video Summarization and Moment Localization by Cross-Task Sample Transfer
To address it, this work explores a new solution for video summarization by transferring samples from a correlated task (i. e., video moment localization) equipped with abundant training data.
Ranked #4 on
Supervised Video Summarization
on SumMe
(using extra training data)
Rethinking the Spatial Route Prior in Vision-and-Language Navigation
In a most information-rich case of knowing environment maps and admitting shortest-path prior, we observe that given an origin-destination node pair, the internal route can be uniquely determined.
Poisoning MorphNet for Clean-Label Backdoor Attack to Point Clouds
To this end, MorphNet jointly optimizes two objectives for sample-adaptive poisoning: a reconstruction loss that preserves the visual similarity between benign / poisoned point clouds, and a classification loss that enforces a modern recognition model of point clouds tends to mis-classify the poisoned sample to a pre-specified target category.
Fast Fourier Convolution
FFC is a generic operator that can directly replace vanilla convolutions in a large body of existing networks, without any adjustments and with comparable complexity metrics (e. g., FLOPs).
Informative Dropout for Robust Representation Learning: A Shape-bias Perspective
Specifically, we discriminate texture from shape based on local self-information in an image, and adopt a Dropout-like algorithm to decorrelate the model output from the local texture.
Weakly-Supervised Action Localization by Generative Attention Modeling
By maximizing the conditional probability with respect to the attention, the action and non-action frames are well separated.
Ranked #11 on
Weakly Supervised Action Localization
on ActivityNet-1.2
Weakly Supervised Action Localization
Weakly-supervised Temporal Action Localization
+1
Fast Non-Local Neural Networks with Spectral Residual Learning
We show its equivalence to conducting residual learning in some spectral domain and carefully re-formulate a variety of neural layers into their spectral forms, such as ReLU or convolutions.
Deep High-Resolution Representation Learning for Visual Recognition
High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection.
Ranked #1 on
Object Detection
on COCO test-dev
(Hardware Burden metric)
Scale Matters: Temporal Scale Aggregation Network for Precise Action Localization in Untrimmed Videos
Our proposed TSA-Net demonstrates clear and consistent better performances and re-calibrates new state-of-the-art on both benchmarks.
Two-Stream Video Classification with Cross-Modality Attention
In the experiments, we comprehensively compare our method with two-stream and non-local models widely used in video classification.
Ranked #35 on
Action Recognition
on UCF101
Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation
Few-shot learning is a nascent research topic, motivated by the fact that traditional deep learning methods require tremen- dous amounts of data.
Ranked #1 on
Few-Shot Semantic Segmentation
on Pascal5i
High-Resolution Representations for Labeling Pixels and Regions
The proposed approach achieves superior results to existing single-model networks on COCO object detection.
Ranked #7 on
Semantic Segmentation
on LIP val
Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues
There are multiple fronts to these endeavors, including object detection on roads, 3-D reconstruction etc., but in this work we focus on a vision-based model that directly maps raw input images to steering angles using deep networks.
Deep Hashing: A Joint Approach for Image Signature Learning
In this paper, we propose a novel algorithm that concurrently performs feature engineering and non-linear supervised hashing function learning.
Learning Binary Codes and Binary Weights for Efficient Classification
The optimization alternatively proceeds over the binary classifiers and image hash codes.
Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization
Stochastic gradient descent (SGD) holds as a classical method to build large scale machine learning models over big data.
Hash-SVM: Scalable Kernel Machines for Large-Scale Visual Classification
This paper presents a novel algorithm which uses compact hash bits to greatly improve the efficiency of non-linear kernel SVM in very large scale visual classification problems.
Distributed Low-rank Subspace Segmentation
Vision problems ranging from image clustering to motion segmentation to semi-supervised learning can naturally be framed as subspace segmentation problems, in which one aims to recover multiple low-dimensional subspaces from noisy and corrupted input data.
