Search Results for author:
Found 37 papers, 11 papers with code
SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory
The Segment Anything Model 2 (SAM 2) has demonstrated strong performance in object segmentation tasks but faces challenges in visual object tracking, particularly when managing crowded scenes with fast-moving or self-occluding objects.
Ranked #1 on
Visual Object Tracking
on GOT-10k
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound
Multimodal Large Language Model (MLLM) has recently garnered attention as a prominent research focus.
Ego3DT: Tracking Every 3D Object in Ego-centric Videos
Addressing this issue, this paper introduces a novel zero-shot approach for the 3D reconstruction and tracking of all objects from the ego-centric video.
PAD: Personalized Alignment of LLMs at Decoding-Time
Aligning with personalized preferences, which vary significantly across cultural, educational, and political differences, poses a significant challenge due to the computational costs and data demands of traditional alignment methods.
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
AuroraCap shows superior performance on various video and image captioning benchmarks, for example, obtaining a CIDEr of 88. 9 on Flickr30k, beating GPT-4V (55. 3) and Gemini-1. 5 Pro (82. 2).
AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement
Existing low-light image enhancement (LIE) methods have achieved noteworthy success in solving synthetic distortions, yet they often fall short in practical applications.
RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark
Traditional methods for human localization and pose estimation (HPE), which mainly rely on RGB images as an input modality, confront substantial limitations in real-world applications due to privacy concerns.
Boosting Online 3D Multi-Object Tracking through Camera-Radar Cross Check
In the domain of autonomous driving, the integration of multi-modal perception techniques based on data from diverse sensors has demonstrated substantial progress.
STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
We begin our exploration with a vanilla large language model, augmenting it with a vision encoder and an action codebase trained on our collected high-quality dataset STEVE-21K.
CityCraft: A Real Crafter for 3D City Generation
The rendered scenes lack variety, resembling the training images, resulting in monotonous styles.
MovieChat+: Question-aware Sparse Memory for Long Video Question Answering
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Ranked #4 on
Question Answering
on NExT-QA (Open-ended VideoQA)
MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection
Subsequently, we introduce the cross-modal residual distillation to transfer the 3D spatial cues.
Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model
After distillation, embodied agents can complete complex, open-ended tasks without additional expert guidance, utilizing the performance and knowledge of a versatile MLM.
VersaT2I: Improving Text-to-Image Models with Versatile Reward
Recent text-to-image (T2I) models have benefited from large-scale and high-quality data, demonstrating impressive performance.
Exploring Learning-based Motion Models in Multi-Object Tracking
In the field of multi-object tracking (MOT), traditional methods often rely on the Kalman Filter for motion prediction, leveraging its strengths in linear motion scenarios.
Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation
To assess organizational behavior, we design a series of navigation tasks in the Minecraft environment, which includes searching and exploring.
Learning Diffusion Texture Priors for Image Restoration
When adopting diffusion models for image restoration the crucial challenge lies in how to preserve high-level image fidelity in the randomness diffusion process and generate accurate background structures and realistic texture details.
User-Aware Prefix-Tuning is a Good Learner for Personalized Image Captioning
Moreover, we employ GPT-2 as the frozen large language model.
CityGen: Infinite and Controllable 3D City Layout Generation
In this paper, we propose CityGen, a novel end-to-end framework for infinite, diverse and controllable 3D city layout generation. First, we propose an outpainting pipeline to extend the local layout to an infinite city layout.
See and Think: Embodied Agent in Virtual Environment
This paper proposes STEVE, a comprehensive and visionary embodied agent in the Minecraft virtual environment.
UniHPE: Towards Unified Human Pose Estimation via Contrastive Learning
In this paper, we propose UniHPE, a unified Human Pose Estimation pipeline, which aligns features from all three modalities, i. e., 2D human pose estimation, lifting-based and image-based 3D human pose estimation, in the same pipeline.
Ranked #70 on
3D Human Pose Estimation
on 3DPW
(PA-MPJPE metric)
Efficient Domain Adaptation via Generative Prior for 3D Infant Pose Estimation
We further apply a guided diffusion model to domain adapt 3D adult pose to infant pose to supplement small datasets.
Devil in the Number: Towards Robust Multi-modality Data Filter
In order to appropriately filter multi-modality data sets on a web-scale, it becomes crucial to employ suitable filtering methods to boost performance and reduce training costs.
Chasing Consistency in Text-to-3D Generation from a Single Image
In light of the above issues, we present Consist3D, a three-stage framework Chasing for semantic-, geometric-, and saturation-Consistent Text-to-3D generation from a single image, in which the first two stages aim to learn parameterized consistency tokens, and the last stage is for optimization.
StableVideo: Text-driven Consistency-aware Diffusion Video Editing
In this paper, we tackle this problem by introducing temporal dependency to existing text-driven diffusion models, which allows them to generate consistent appearance for the edited objects.
PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation
Typically, PoSynDA uses a diffusion-inspired structure to simulate 3D pose distribution in the target domain.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Video-based Generative Performance Benchmarking (Consistency)
Video-based Generative Performance Benchmarking (Contextual Understanding)
+10
Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation
Learning-based methods have dominated the 3D human pose estimation (HPE) tasks with significantly better performance in most benchmarks than traditional optimization-based methods.
Ranked #10 on
3D Human Pose Estimation
on 3DPW
(PA-MPJPE metric)
A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision
Deep learning has the potential to revolutionize sports performance, with applications ranging from perception and comprehension to decision.
MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Estimating 3D human poses only from a 2D human pose sequence is thoroughly explored in recent years.
Five A$^{+}$ Network: You Only Need 9K Parameters for Underwater Image Enhancement
In this work, we propose the Five A$^{+}$ Network (FA$^{+}$Net), a highly efficient and lightweight real-time underwater image enhancement network with only $\sim$ 9k parameters and $\sim$ 0. 01s processing time.
Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation
We observe that the degradation is caused by two factors: 1) the large distribution gap over global positions of poses between the source and target datasets due to variant camera parameters and settings, and 2) the deficient diversity of local structures of poses in training.
3D Human Pose Estimation
3D Human Pose Estimation in Limited Data
+4
Blind Inpainting with Object-aware Discrimination for Artificial Marker Removal
In this paper, we propose a novel blind inpainting method that automatically reconstructs visual contents within the corrupted regions without mask input as guidance.
Deep Learning Methods for Small Molecule Drug Discovery: A Survey
With the development of computer-assisted techniques, research communities including biochemistry and deep learning have been devoted into the drug discovery field for over a decade.
DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models
We focus on a new fashion design task, where we aim to transfer a reference appearance image onto a clothing image while preserving the structure of the clothing image.
Weakly Supervised Two-Stage Training Scheme for Deep Video Fight Detection Model
In this paper, we propose a simple but effective method that solves the task from a new perspective: we design the fight detection model as a composition of an action-aware feature extractor and an anomaly score generator.
