Search Results for author:
Found 35 papers, 15 papers with code
Sign Language Video Anonymization
We modify a motion-based image animation model to generate high-resolution videos with the signer identity changed, but with the preservation of linguistically significant motions and facial expressions.
LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generation
Large foundation models trained on large-scale visual-text data can significantly enhance Open Vocabulary Object Detection (OVD) through data generation.
STRIDE: Automating Reward Design, Deep Reinforcement Learning Training and Feedback Optimization in Humanoid Robotics Locomotion
Humanoid robotics presents significant challenges in artificial intelligence, requiring precise coordination and control of high-degree-of-freedom systems.
The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering
Extensive experiments show that VISTA on average reduces hallucination by abount 40% on evaluated open-ended generation task, and it consistently outperforms existing methods on four benchmarks across four architectures under three decoding strategies.
STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes
We present STORM, a spatio-temporal reconstruction model designed for reconstructing dynamic outdoor scenes from sparse observations.
DreamDrive: Generative 4D Scene Modeling from Street View Images
In this paper, we present DreamDrive, a 4D spatial-temporal scene generation approach that combines the merits of generation and reconstruction, to synthesize generalizable 4D driving scenes and dynamic driving videos with 3D consistency.
Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models
Traffic simulation aims to learn a policy for traffic agents that, when unrolled in closed-loop, faithfully recovers the joint distribution of trajectories observed in the real world.
Exploiting VLM Localizability and Semantics for Open Vocabulary Action Detection
Existing approaches focus on the closed-set setting where an action detector is trained and tested on videos from a fixed set of action categories.
Optimal Defenses Against Gradient Reconstruction Attacks
Federated Learning (FL) is designed to prevent data leakage through collaborative model training without centralized data storage.
Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment
Learning to localize temporal boundaries of procedure steps in instructional videos is challenging due to the limited availability of annotated large-scale training videos.
Promptable Closed-loop Traffic Simulation
Simulation stands as a cornerstone for safe and efficient autonomous driving development.
Wolf: Captioning Everything with a World Summarization Framework
Finally, we establish a benchmark for video captioning and introduce a leaderboard, aiming to accelerate advancements in video understanding, captioning, and data alignment.
Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
The autonomous driving industry is increasingly adopting end-to-end learning from sensory inputs to minimize human biases in system design.
Data Reconstruction Attacks and Defenses: A Systematic Evaluation
Reconstruction attacks and defenses are essential in understanding the data leakage problem in machine learning.
Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent
Adept traffic models are critical to both planning and closed-loop simulation for autonomous vehicles (AV), and key design objectives include accuracy, diverse multimodal behaviors, interpretability, and downstream compatibility.
Interactive Joint Planning for Autonomous Vehicles
In particular, IJP jointly optimizes over the behavior of the ego and the surrounding agents and leverages deep-learned prediction models as prediction priors that the join trajectory optimization tries to stay close to.
Language-Guided Traffic Simulation via Scene-Level Diffusion
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development.
Improving Tuning-Free Real Image Editing with Proximal Guidance
Null-text inversion (NTI) optimizes null embeddings to align the reconstruction and inversion trajectories with larger CFG scales, enabling real image editing with cross-attention control.
Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens
However, direct aligning cross-modal information using such representations is challenging, as visual patches and text tokens differ in semantic levels and granularities.
HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention
The success of large-scale contrastive vision-language pretraining (CLIP) has benefited both visual recognition and multimodal content understanding.
Guided Conditional Diffusion for Controllable Traffic Simulation
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles.
BITS: Bi-level Imitation for Traffic Simulation
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets and show that BITS achieves balanced traffic simulation performance in realism, diversity, and long-horizon stability.
Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learning
Despite the success of fully-supervised human skeleton sequence modeling, utilizing self-supervised pre-training for skeleton sequence representation learning has been an active field because acquiring task-specific skeleton annotations at large scales is difficult.
ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning
In this work, we present ScePT, a policy planning-based trajectory prediction model that generates accurate, scene-consistent trajectory predictions suitable for autonomous system motion planning.
Onboard Safety Guarantees for Racing Drones: High-speed Geofencing with Control Barrier Functions
This paper details the theory and implementation behind practically ensuring safety of remotely piloted racing drones.
Interactive multi-modal motion planning with Branch Model Predictive Control
Motion planning for autonomous robots and vehicles in presence of uncontrolled agents remains a challenging problem as the reactive behaviors of the uncontrolled agents must be considered.
Density Constrained Reinforcement Learning
We study constrained reinforcement learning (CRL) from a novel perspective by setting constraints directly on state density functions, rather than the value functions considered by previous works.
More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matching
Cross-modal attention mechanisms have been widely applied to the image-text matching task and have achieved remarkable improvements thanks to its capability of learning fine-grained relevance across different modalities.
Backup Control Barrier Functions: Formulation and Comparative Study
The backup control barrier function (CBF) was recently proposed as a tractable formulation that guarantees the feasibility of the CBF quadratic programming (QP) via an implicitly defined control invariant set.
Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates
We propose a novel joint-learning framework that can be implemented in a decentralized fashion, with generalization guarantees for certain function classes.
Reactive motion planning with probabilistic safety guarantees
Motion planning in environments with multiple agents is critical to many important autonomous applications such as autonomous vehicles and assistive robots.
Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge
Our key idea is to generalize the distilled cross-modal knowledge learned from a Source dataset, which contains paired examples from both modalities, to the Target dataset by modeling knowledge as priors on parameters of the Student.
Counter-example Guided Learning of Bounds on Environment Behavior
There is a growing interest in building autonomous systems that interact with complex environments.
Construct Dynamic Graphs for Hand Gesture Recognition via Spatial-Temporal Attention
We propose a Dynamic Graph-Based Spatial-Temporal Attention (DG-STA) method for hand gesture recognition.
Ranked #4 on
Hand Gesture Recognition
on SHREC 2017
Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM
Sentiment analysis on large-scale social media data is important to bridge the gaps between social media contents and real world activities including political election prediction, individual and public emotional status monitoring and analysis, and so on.
