Search Results for author:
Found 47 papers, 16 papers with code
Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model
After distillation, embodied agents can complete complex, open-ended tasks without additional expert guidance, utilizing the performance and knowledge of a versatile MLM.
VersaT2I: Improving Text-to-Image Models with Versatile Reward
Recent text-to-image (T2I) models have benefited from large-scale and high-quality data, demonstrating impressive performance.
Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation
To assess organizational behavior, we design a series of navigation tasks in the Minecraft environment, which includes searching and exploring.
MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant
Medical generative models, acknowledged for their high-quality sample generation ability, have accelerated the fast growth of medical applications.
UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts
Medical vision-language pre-training (Med-VLP) models have recently accelerated the fast-growing medical diagnostics application.
User-Aware Prefix-Tuning is a Good Learner for Personalized Image Captioning
Moreover, we employ GPT-2 as the frozen large language model.
CityGen: Infinite and Controllable 3D City Layout Generation
In this paper, we propose CityGen, a novel end-to-end framework for infinite, diverse and controllable 3D city layout generation. First, we propose an outpainting pipeline to extend the local layout to an infinite city layout.
See and Think: Embodied Agent in Virtual Environment
Vision perception involves the interpretation of visual information in the environment, which is then integrated into the LLMs component with agent state and task instruction.
Vision meets mmWave Radar: 3D Object Perception Benchmark for Autonomous Driving
This kind of radar format can enable machine learning models to generate more reliable object perception results after interacting and fusing the information or features between the camera and radar.
Sam-Guided Enhanced Fine-Grained Encoding with Mixed Semantic Learning for Medical Image Captioning
With the development of multimodality and large language models, the deep learning-based technique for medical image captioning holds the potential to offer valuable diagnostic recommendations.
Devil in the Number: Towards Robust Multi-modality Data Filter
In order to appropriately filter multi-modality data sets on a web-scale, it becomes crucial to employ suitable filtering methods to boost performance and reduce training costs.
FrameRS: A Video Frame Compression Model Composed by Self supervised Video Frame Reconstructor and Key Frame Selector
The key frame selector, Frame Selector, is built on CNN architecture.
Chasing Consistency in Text-to-3D Generation from a Single Image
In light of the above issues, we present Consist3D, a three-stage framework Chasing for semantic-, geometric-, and saturation-Consistent Text-to-3D generation from a single image, in which the first two stages aim to learn parameterized consistency tokens, and the last stage is for optimization.
Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection
Thus, the optimum of the distillation loss does not necessarily lead to the optimal student classification scores for dense object detectors.
UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning
Our proposed model takes support images and labels as prompt guidance for a query image.
StableVideo: Text-driven Consistency-aware Diffusion Video Editing
In this paper, we tackle this problem by introducing temporal dependency to existing text-driven diffusion models, which allows them to generate consistent appearance for the edited objects.
PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation
Typically, PoSynDA uses a diffusion-inspired structure to simulate 3D pose distribution in the target domain.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Video-based Generative Performance Benchmarking (Consistency)
Video-based Generative Performance Benchmarking (Contextual Understanding)
+10
A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision
Deep learning has the potential to revolutionize sports performance, with applications ranging from perception and comprehension to decision.
MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
In this paper, we propose MPM, a unified 2D-3D human pose representation framework via masked pose modeling.
Language Adaptive Weight Generation for Multi-task Visual Grounding
The active perception can take expressions as priors to extract relevant visual features, which can effectively alleviate the mismatches.
SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation
Experimental results on Stanford2D3D Panoramic datasets show that SGAT4PASS significantly improves performance and robustness, with approximately a 2% increase in mIoU, and when small 3D disturbances occur in the data, the stability of our performance is improved by an order of magnitude.
Ranked #4 on
Semantic Segmentation
on Stanford2D3D Panoramic
Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation
We observe that the degradation is caused by two factors: 1) the large distribution gap over global positions of poses between the source and target datasets due to variant camera parameters and settings, and 2) the deficient diversity of local structures of poses in training.
3D Human Pose Estimation
3D Human Pose Estimation in Limited Data
+3
DDMM-Synth: A Denoising Diffusion Model for Cross-modal Medical Image Synthesis with Sparse-view Measurement Embedding
Reducing the radiation dose in computed tomography (CT) is important to mitigate radiation-induced risks.
Blind Inpainting with Object-aware Discrimination for Artificial Marker Removal
The reconstruction network consists of two branches that predict the corrupted regions with artificial markers and simultaneously recover the missing visual contents.
Deep Learning Methods for Small Molecule Drug Discovery: A Survey
With the development of computer-assisted techniques, research communities including biochemistry and deep learning have been devoted into the drug discovery field for over a decade.
DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes
Although cross-view multi-object tracking has received increased attention in recent years, existing datasets still have several issues, including 1) missing real-world scenarios, 2) lacking diverse scenes, 3) owning a limited number of tracks, 4) comprising only static cameras, and 5) lacking standard benchmarks, which hinder the investigation and comparison of cross-view tracking methods.
DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models
We focus on a new fashion design task, where we aim to transfer a reference appearance image onto a clothing image while preserving the structure of the clothing image.
STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks
We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks.
Ranked #3 on
Audio Classification
on SHD
Missing Modality meets Meta Sampling (M3S): An Efficient Universal Approach for Multimodal Sentiment Analysis with Missing Modality
Multimodal sentiment analysis (MSA) is an important way of observing mental activities with the help of data captured from multiple modalities.
Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection
Anomaly detection aims at identifying deviant samples from the normal data distribution.
Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey
Multi-object tracking (MOT) aims to associate target objects across video frames in order to obtain entire moving trajectories.
Preserve Pre-trained Knowledge: Transfer Learning With Self-Distillation For Action Recognition
Video-based action recognition is one of the most popular topics in computer vision.
Human-Centered Prior-Guided and Task-Dependent Multi-Task Representation Learning for Action Recognition Pre-Training
Recently, much progress has been made for self-supervised action recognition.
Self-paced Multi-grained Cross-modal Interaction Modeling for Referring Expression Comprehension
As an important and challenging problem in vision-language tasks, referring expression comprehension (REC) generally requires a large amount of multi-grained information of visual and linguistic modalities to realize accurate reasoning.
MAP-SNN: Mapping Spike Activities with Multiplicity, Adaptability, and Plasticity into Bio-Plausible Spiking Neural Networks
For plasticity, we propose a trainable convolutional synapse that models spike response current to enhance the diversity of spiking neurons for temporal feature extraction.
Ranked #6 on
Audio Classification
on SHD
Disjoint Contrastive Regression Learning for Multi-Sourced Annotations
However, the inconsistency and bias among different annotators are harmful to the model training, especially for qualitative and subjective tasks. To address this challenge, in this paper, we propose a novel contrastive regression framework to address the disjoint annotations problem, where each sample is labeled by only one annotator and multiple annotators work on disjoint subsets of the data.
ActiveMatch: End-to-end Semi-supervised Active Representation Learning
With human-in-the-loop, active learning can iteratively select informative unlabeled samples for labeling and training to improve the performance in the SSL framework.
Track without Appearance: Learn Box and Tracklet Embedding with Local and Global Motion Patterns for Vehicle Tracking
In this paper, we try to explore the significance of motion patterns for vehicle tracking without appearance information.
Rethinking of Radar's Role: A Camera-Radar Dataset and Systematic Annotator via Coordinate Alignment
Radar has long been a common sensor on autonomous vehicles for obstacle ranging and speed estimation.
Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking
To address such common challenges in most of the existing trackers, in this paper, a tracklet booster algorithm is proposed, which can be built upon any other tracker.
When Few-Shot Learning Meets Video Object Detection
We employ a transfer-learning framework to effectively train the video object detector on a large number of base-class objects and a few video clips of novel-class objects.
DAIL: Dataset-Aware and Invariant Learning for Face Recognition
To solve the first issue of identity overlapping, we propose a dataset-aware loss for multi-dataset training by reducing the penalty when the same person appears in multiple datasets.
Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution
Videos that contain multiple potentially occluded people captured from freely moving monocular cameras are very common in real-world scenarios, while 3D HPE for such scenarios is quite challenging, partially because there is a lack of such data with accurate 3D ground truth labels in existing datasets.
Eye in the Sky: Drone-Based Object Tracking and 3D Localization
Drones, or general UAVs, equipped with a single camera have been widely deployed to a broad range of applications, such as aerial photography, fast goods delivery and most importantly, surveillance.
Exploit the Connectivity: Multi-Object Tracking with TrackletNet
Multi-object tracking (MOT) is an important and practical task related to both surveillance systems and moving camera applications, such as autonomous driving and robotic vision.
Ranked #19 on
Multi-Object Tracking
on MOT16
Multiple-Kernel Based Vehicle Tracking Using 3D Deformable Model and Camera Self-Calibration
In this challenge, we propose a model-based vehicle localization method, which builds a kernel at each patch of the 3D deformable vehicle model and associates them with constraints in 3D space.
