Search Results for author:
Found 62 papers, 20 papers with code
EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environments
Developing autonomous home robots controlled by natural language has long been a pursuit of human.
Pointmap Association and Piecewise-Plane Constraint for Consistent and Compact 3D Gaussian Segmentation Field
To address these challenges, we introduce CCGS, a method designed to achieve both view consistent 2D segmentation and a compact 3D Gaussian segmentation field.
Efficient Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment
Spiking Neural Networks (SNNs) are emerging as a brain-inspired alternative to traditional Artificial Neural Networks (ANNs), prized for their potential energy efficiency on neuromorphic hardware.
RDG-GS: Relative Depth Guidance with Gaussian Splatting for Real-time Sparse-View 3D Rendering
While advanced techniques like radiance fields and 3D Gaussian Splatting achieve rendering quality and impressive efficiency with dense view inputs, they suffer from significant geometric reconstruction errors when applied to sparse input views.
BioTD: an online database of biotoxins
Biotoxins, mainly produced by venomous animals, plants and microorganisms, exhibit high physiological activity and unique effects such as lowering blood pressure and analgesia.
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound
Multimodal Large Language Model (MLLM) has recently garnered attention as a prominent research focus.
Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation
Recent insights have revealed that rate-coding is a primary form of information representation captured by surrogate-gradient-based Backpropagation Through Time (BPTT) in training deep Spiking Neural Networks (SNNs).
Ego3DT: Tracking Every 3D Object in Ego-centric Videos
Addressing this issue, this paper introduces a novel zero-shot approach for the 3D reconstruction and tracking of all objects from the ego-centric video.
Frame Order Matters: A Temporal Sequence-Aware Model for Few-Shot Action Recognition
In this paper, we propose a novel Temporal Sequence-Aware Model (TSAM) for few-shot action recognition (FSAR), which incorporates a sequential perceiver adapter into the pre-training framework, to integrate both the spatial information and the sequential temporal dynamics into the feature embeddings.
STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
We begin our exploration with a vanilla large language model, augmenting it with a vision encoder and an action codebase trained on our collected high-quality dataset STEVE-21K.
BEVSpread: Spread Voxel Pooling for Bird's-Eye-View Representation in Vision-based Roadside 3D Object Detection
Vision-based roadside 3D object detection has attracted rising attention in autonomous driving domain, since it encompasses inherent advantages in reducing blind spots and expanding perception range.
CityCraft: A Real Crafter for 3D City Generation
The rendered scenes lack variety, resembling the training images, resulting in monotonous styles.
S4Fusion: Saliency-aware Selective State Space Model for Infrared Visible Image Fusion
However, in image fusion, current methods underestimate the potential of SSSM in capturing the global spatial information of both modalities.
FlexiFilm: Long Video Generation with Flexible Conditions
Generating long and consistent videos has emerged as a significant yet challenging problem.
MovieChat+: Question-aware Sparse Memory for Long Video Question Answering
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Ranked #4 on
Question Answering
on NExT-QA (Open-ended VideoQA)
Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model
After distillation, embodied agents can complete complex, open-ended tasks without additional expert guidance, utilizing the performance and knowledge of a versatile MLM.
VersaT2I: Improving Text-to-Image Models with Versatile Reward
Recent text-to-image (T2I) models have benefited from large-scale and high-quality data, demonstrating impressive performance.
Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation
To assess organizational behavior, we design a series of navigation tasks in the Minecraft environment, which includes searching and exploring.
MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant
Medical generative models, acknowledged for their high-quality sample generation ability, have accelerated the fast growth of medical applications.
UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts
Medical vision-language pre-training (Med-VLP) models have recently accelerated the fast-growing medical diagnostics application.
User-Aware Prefix-Tuning is a Good Learner for Personalized Image Captioning
Moreover, we employ GPT-2 as the frozen large language model.
CityGen: Infinite and Controllable 3D City Layout Generation
In this paper, we propose CityGen, a novel end-to-end framework for infinite, diverse and controllable 3D city layout generation. First, we propose an outpainting pipeline to extend the local layout to an infinite city layout.
See and Think: Embodied Agent in Virtual Environment
This paper proposes STEVE, a comprehensive and visionary embodied agent in the Minecraft virtual environment.
Vision meets mmWave Radar: 3D Object Perception Benchmark for Autonomous Driving
This kind of radar format can enable machine learning models to generate more reliable object perception results after interacting and fusing the information or features between the camera and radar.
Sam-Guided Enhanced Fine-Grained Encoding with Mixed Semantic Learning for Medical Image Captioning
With the development of multimodality and large language models, the deep learning-based technique for medical image captioning holds the potential to offer valuable diagnostic recommendations.
Devil in the Number: Towards Robust Multi-modality Data Filter
In order to appropriately filter multi-modality data sets on a web-scale, it becomes crucial to employ suitable filtering methods to boost performance and reduce training costs.
FrameRS: A Video Frame Compression Model Composed by Self supervised Video Frame Reconstructor and Key Frame Selector
The key frame selector, Frame Selector, is built on CNN architecture.
Chasing Consistency in Text-to-3D Generation from a Single Image
In light of the above issues, we present Consist3D, a three-stage framework Chasing for semantic-, geometric-, and saturation-Consistent Text-to-3D generation from a single image, in which the first two stages aim to learn parameterized consistency tokens, and the last stage is for optimization.
Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection
Thus, the optimum of the distillation loss does not necessarily lead to the optimal student classification scores for dense object detectors.
PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation
Typically, PoSynDA uses a diffusion-inspired structure to simulate 3D pose distribution in the target domain.
StableVideo: Text-driven Consistency-aware Diffusion Video Editing
In this paper, we tackle this problem by introducing temporal dependency to existing text-driven diffusion models, which allows them to generate consistent appearance for the edited objects.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks.
Multiple-choice
Video-based Generative Performance Benchmarking (Consistency)
+11
A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision
Deep learning has the potential to revolutionize sports performance, with applications ranging from perception and comprehension to decision.
MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Estimating 3D human poses only from a 2D human pose sequence is thoroughly explored in recent years.
Language Adaptive Weight Generation for Multi-task Visual Grounding
The active perception can take expressions as priors to extract relevant visual features, which can effectively alleviate the mismatches.
SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation
Experimental results on Stanford2D3D Panoramic datasets show that SGAT4PASS significantly improves performance and robustness, with approximately a 2% increase in mIoU, and when small 3D disturbances occur in the data, the stability of our performance is improved by an order of magnitude.
Ranked #4 on
Semantic Segmentation
on Stanford2D3D Panoramic
Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation
We observe that the degradation is caused by two factors: 1) the large distribution gap over global positions of poses between the source and target datasets due to variant camera parameters and settings, and 2) the deficient diversity of local structures of poses in training.
3D Human Pose Estimation
3D Human Pose Estimation in Limited Data
+4
DDMM-Synth: A Denoising Diffusion Model for Cross-modal Medical Image Synthesis with Sparse-view Measurement Embedding
Reducing the radiation dose in computed tomography (CT) is important to mitigate radiation-induced risks.
Blind Inpainting with Object-aware Discrimination for Artificial Marker Removal
In this paper, we propose a novel blind inpainting method that automatically reconstructs visual contents within the corrupted regions without mask input as guidance.
Deep Learning Methods for Small Molecule Drug Discovery: A Survey
With the development of computer-assisted techniques, research communities including biochemistry and deep learning have been devoted into the drug discovery field for over a decade.
DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes
Although cross-view multi-object tracking has received increased attention in recent years, existing datasets still have several issues, including 1) missing real-world scenarios, 2) lacking diverse scenes, 3) owning a limited number of tracks, 4) comprising only static cameras, and 5) lacking standard benchmarks, which hinder the investigation and comparison of cross-view tracking methods.
DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models
We focus on a new fashion design task, where we aim to transfer a reference appearance image onto a clothing image while preserving the structure of the clothing image.
STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks
We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks.
Ranked #4 on
Audio Classification
on SHD
Missing Modality meets Meta Sampling (M3S): An Efficient Universal Approach for Multimodal Sentiment Analysis with Missing Modality
Multimodal sentiment analysis (MSA) is an important way of observing mental activities with the help of data captured from multiple modalities.
Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection
Anomaly detection aims at identifying deviant samples from the normal data distribution.
Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey
Multi-object tracking (MOT) aims to associate target objects across video frames in order to obtain entire moving trajectories.
Preserve Pre-trained Knowledge: Transfer Learning With Self-Distillation For Action Recognition
Video-based action recognition is one of the most popular topics in computer vision.
Human-Centered Prior-Guided and Task-Dependent Multi-Task Representation Learning for Action Recognition Pre-Training
Recently, much progress has been made for self-supervised action recognition.
Self-paced Multi-grained Cross-modal Interaction Modeling for Referring Expression Comprehension
As an important and challenging problem in vision-language tasks, referring expression comprehension (REC) generally requires a large amount of multi-grained information of visual and linguistic modalities to realize accurate reasoning.
MAP-SNN: Mapping Spike Activities with Multiplicity, Adaptability, and Plasticity into Bio-Plausible Spiking Neural Networks
For plasticity, we propose a trainable convolutional synapse that models spike response current to enhance the diversity of spiking neurons for temporal feature extraction.
Ranked #7 on
Audio Classification
on SHD
Disjoint Contrastive Regression Learning for Multi-Sourced Annotations
However, the inconsistency and bias among different annotators are harmful to the model training, especially for qualitative and subjective tasks. To address this challenge, in this paper, we propose a novel contrastive regression framework to address the disjoint annotations problem, where each sample is labeled by only one annotator and multiple annotators work on disjoint subsets of the data.
ActiveMatch: End-to-end Semi-supervised Active Representation Learning
With human-in-the-loop, active learning can iteratively select informative unlabeled samples for labeling and training to improve the performance in the SSL framework.
Track without Appearance: Learn Box and Tracklet Embedding with Local and Global Motion Patterns for Vehicle Tracking
In this paper, we try to explore the significance of motion patterns for vehicle tracking without appearance information.
Rethinking of Radar's Role: A Camera-Radar Dataset and Systematic Annotator via Coordinate Alignment
Radar has long been a common sensor on autonomous vehicles for obstacle ranging and speed estimation.
Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking
To address such common challenges in most of the existing trackers, in this paper, a tracklet booster algorithm is proposed, which can be built upon any other tracker.
When Few-Shot Learning Meets Video Object Detection
We employ a transfer-learning framework to effectively train the video object detector on a large number of base-class objects and a few video clips of novel-class objects.
DAIL: Dataset-Aware and Invariant Learning for Face Recognition
To solve the first issue of identity overlapping, we propose a dataset-aware loss for multi-dataset training by reducing the penalty when the same person appears in multiple datasets.
Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution
Videos that contain multiple potentially occluded people captured from freely moving monocular cameras are very common in real-world scenarios, while 3D HPE for such scenarios is quite challenging, partially because there is a lack of such data with accurate 3D ground truth labels in existing datasets.
Eye in the Sky: Drone-Based Object Tracking and 3D Localization
Drones, or general UAVs, equipped with a single camera have been widely deployed to a broad range of applications, such as aerial photography, fast goods delivery and most importantly, surveillance.
Exploit the Connectivity: Multi-Object Tracking with TrackletNet
Multi-object tracking (MOT) is an important and practical task related to both surveillance systems and moving camera applications, such as autonomous driving and robotic vision.
Ranked #20 on
Multi-Object Tracking
on MOT16
Multiple-Kernel Based Vehicle Tracking Using 3D Deformable Model and Camera Self-Calibration
In this challenge, we propose a model-based vehicle localization method, which builds a kernel at each patch of the 3D deformable vehicle model and associates them with constraints in 3D space.
