Search Results for author:
Found 91 papers, 75 papers with code
MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model
In this paper, we propose a novel LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model, termed MambaMOS.
Skeleton-Based Human Action Recognition with Noisy Labels
In this study, we bridge this gap by implementing a framework that augments well-established skeleton-based human action recognition methods with label-denoising strategies from various research areas to serve as the initial benchmark.
Real-World Computational Aberration Correction via Quantized Domain-Mixing Representation
Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications.
OccFiner: Offboard Occupancy Refinement with Hybrid Propagation
Vision-based occupancy prediction, also known as 3D Semantic Scene Completion (SSC), presents a significant challenge in computer vision.
EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving
This paper introduces the task of Auditory Referring Multi-Object Tracking (AR-MOT), which dynamically tracks specific objects in a video sequence based on audio expressions and appears as a challenging problem in autonomous driving.
Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers
Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content.
LF Tracy: A Unified Single-Pipeline Approach for Salient Object Detection in Light Field Cameras
Previous approaches predominantly employ a custom two-stream design to discover the implicit angular feature within light field cameras, leading to significant information isolation between different LF representations.
Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation
Integrating information from multiple modalities enhances the robustness of scene perception systems in autonomous vehicles, providing a more comprehensive and reliable sensory framework.
Navigating Open Set Scenarios for Skeleton-based Action Recognition
In real-world scenarios, human actions often fall outside the distribution of training data, making it crucial for models to recognize known actions and reject unknown ones.
Rethinking Event-based Human Pose Estimation with 3D Event Representations
Experiments on EV-3DPW demonstrate that the robustness of our proposed 3D representation methods compared to traditional RGB images and event frame techniques under the same backbones.
CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity
Roadside camera-driven 3D object detection is a crucial task in intelligent transportation systems, which extends the perception range beyond the limitations of vision-centric vehicles and enhances road safety.
Ranked #2 on
3D Object Detection
on Rope3D
Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision
These works overlooked the differences in performance among modalities, which led to the propagation of erroneous knowledge between modalities while only three fundamental modalities, i. e., joints, bones, and motions are used, hence no additional modalities are explored.
Unveiling the Hidden Realm: Self-supervised Skeleton-based Action Recognition in Occluded Environments
To integrate action recognition methods into autonomous robotic systems, it is crucial to consider adverse situations involving target occlusions.
S$^3$-MonoDETR: Supervised Shape&Scale-perceptive Deformable Transformer for Monocular 3D Object Detection
These methods typically use visual and depth representations to generate query points on objects, whose quality plays a decisive role in the detection accuracy.
FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving
Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point Control Loss (CPCL) function for diverse point-wise supervision; 2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition Control Encoder (CCE).
EPCFormer: Expression Prompt Collaboration Transformer for Universal Referring Video Object Segmentation
Next, we propose an Expression Alignment (EA) mechanism for audio and text expressions.
Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation
The recently rising markup-to-image generation poses greater challenges as compared to natural image generation, due to its low tolerance for errors as well as the complex sequence and context correlations between markup and rendered image.
OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation
To avoid feature loss during network propagation and simultaneously streamline the redundant information from the light field camera, we present a simple yet very effective Sub-Aperture Fusion Module (SAFM) to embed sub-aperture images into angular features without any additional memory cost.
Tightly-Coupled LiDAR-Visual SLAM Based on Geometric Features for Mobile Agents
On the other hand, the entire line segment detected by the visual subsystem overcomes the limitation of the LiDAR subsystem, which can only perform the local calculation for geometric features.
Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairments
Grounded Situation Recognition (GSR) is capable of recognizing and interpreting visual scenes in a contextually intuitive way, yielding salient activities (verbs) and the involved entities (roles) depicted in images.
Towards Anytime Optical Flow Estimation with Event Cameras
We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision.
Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers
In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to address minimalist and high-quality panoramic imaging.
LF-PGVIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras using Points and Geodesic Segments
The purpose of our research is to unleash the potential of point-line odometry with large-FoV omnidirectional cameras, even for cameras with negative-plane FoV.
VPUFormer: Visual Prompt Unified Transformer for Interactive Image Segmentation
Specifically, we design a Prompt-unified Encoder (PuE) by using Gaussian mapping to generate a unified one-dimensional vector for click, box, and scribble prompts, which well captures users' intentions as well as provides a denser representation of user prompts.
Towards Source-free Domain Adaptive Semantic Segmentation via Importance-aware and Prototype-contrast Learning
It utilizes a well-trained source model and unlabeled target data to achieve adaptation in the target domain.
RelaMiX: Exploring Few-Shot Adaptation in Video-based Action Recognition
Domain adaptation is essential for activity recognition to ensure accurate and robust performance across diverse environments, sensor types, and data sources.
SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection
To tackle this problem, this paper proposes a novel "Supervised Scale-aware Deformable Attention" (SSDA) for monocular 3D object detection.
Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving
At the same time, an Across-Space Loss (ASL) is designed to mitigate the negative impact of geometric distortions.
AdaptiveClick: Clicks-aware Transformer with Adaptive Focal Loss for Interactive Image Segmentation
Interactive Image Segmentation (IIS) has emerged as a promising technique for decreasing annotation time.
FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation
This paper raises the new task of Fisheye Semantic Completion (FSC), where dense texture, structure, and semantics of a fisheye image are inferred even beyond the sensor field-of-view (FoV).
PanoVPR: Towards Unified Perspective-to-Equirectangular Visual Place Recognition via Sliding Windows across the Panoramic View
To address this, we propose \textit{PanoVPR}, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping.
360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View
Seeing only a tiny part of the whole is not knowing the full circumstance.
Delivering Arbitrary-Modal Semantic Segmentation
To make this possible, we present the arbitrary cross-modal segmentation model CMNeXt.
Ranked #1 on
Semantic Segmentation
on DSEC
MuscleMap: Towards Video-based Activated Muscle Group Estimation in the Wild
In this paper, we tackle the new task of video-based Activated Muscle Group Estimation (AMGE) aiming at identifying active muscle regions during physical activity in the wild.
MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments
People with Visual Impairments (PVI) typically recognize objects through haptic perception.
Computational Imaging for Machine Perception: Transferring Semantic Segmentation beyond Aberrations
Further, we propose Computational Imaging Assisted Domain Adaptation (CIADA) to leverage prior knowledge of CI for robust performance in SSOA.
Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformer
In this paper, we propose the concept of online video inpainting for autonomous vehicles to expand the field of view, thereby enhancing scene visibility, perception, and system safety.
Ranked #1 on
Seeing Beyond the Visible
on KITTI360-EX
LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents
As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable.
Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation
In this paper, we address panoramic semantic segmentation which is under-explored due to two critical challenges: (1) image distortions and object deformations on panoramas; (2) lack of semantic annotations in the 360-degree imagery.
Ranked #1 on
Semantic Segmentation
on SynPASS
Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers
Humans have an innate ability to sense their surroundings, as they can extract the spatial representation from the egocentric perception and form an allocentric semantic map via spatial transformation and memory updating.
Multi-modal Depression Estimation based on Sub-attentional Fusion
Failure to timely diagnose and effectively treat depression leads to over 280 million people suffering from this psychological disorder worldwide.
Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning
In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to the panoramic domain in a cost-minimizing way.
Annular Computational Imaging: Capture Clear Panoramic Images through Simple Lens
In this paper, we propose an Annular Computational Imaging (ACI) framework to break the optical limit of light-weight PAL design.
Efficient Human Pose Estimation via 3D Event Point Cloud
We then leverage the rasterized event point cloud as input to three different backbones, PointNet, DGCNN, and Point Transformer, with two linear layer decoders to predict the location of human keypoints.
Ranked #1 on
3D Human Pose Estimation
on DHP19
Review on Panoramic Imaging and Its Applications in Scene Understanding
However, while satisfying the need for large-FoV photographic imaging, panoramic imaging instruments are expected to have high resolution, no blind area, miniaturization, and multidimensional intelligent perception, and can be combined with artificial intelligence methods towards the next generation of intelligent instruments, enabling deeper understanding and more holistic perception of 360-degree real-world surrounding environments.
Towards Automatic Parsing of Structured Visual Content through the Use of Synthetic Data
In this work, we propose a synthetic dataset, containing SVCs in the form of images as well as ground truths.
Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates
In this work, we for the first time examine how well the confidence values of modern driver observation models indeed match the probability of the correct outcome and show that raw neural network-based approaches tend to significantly overestimate their prediction quality.
Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor
Then, poses and speed of tracked dynamic objects can be estimated, which are passed to the users through acoustic feedback.
Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning
Autonomous vehicles utilize urban scene segmentation to understand the real world like a human and react accordingly.
Ranked #1 on
Semantic Segmentation
on DADA-seg
(using extra training data)
MatchFormer: Interleaving Attention in Transformers for Feature Matching
While detector-based methods coupled with feature descriptors struggle in low-texture scenes, CNN-based methods with a sequential extract-to-match pipeline, fail to make use of the matching capacity of the encoder and tend to overburden the decoder for matching.
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers
Pixel-wise semantic segmentation of RGB images can be advanced by exploiting complementary features from the supplementary modality (X-modality).
Ranked #1 on
Semantic Segmentation
on SpectralWaste
TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration
This module operates in the latent feature-space enriching and diversifying the training set at feature-level in order to improve generalization to novel data appearances, (e. g., sensor changes) and general feature quality.
Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation
To get around this domain difference and bring together semantic annotations from pinhole- and 360-degree surround-visuals, we propose to learn object deformations and panoramic image distortions in the Deformable Patch Embedding (DPE) and Deformable MLP (DMLP) components which blend into our Transformer for PAnoramic Semantic Segmentation (Trans4PASS) model.
Ranked #2 on
Semantic Segmentation
on SynPASS
TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation
Semantic segmentation benchmarks in the realm of autonomous driving are dominated by large pre-trained transformers, yet their widespread adoption is impeded by substantial computational costs and prolonged training durations.
PanoFlow: Learning 360° Optical Flow for Surrounding Temporal Understanding
PanoFlow achieves state-of-the-art performance on the public OmniFlowNet and the established FlowScape benchmarks.
LF-VIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras with Negative Plane
To tackle this issue, we propose LF-VIO, a real-time VIO framework for cameras with extremely large FoV.
Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions
Yet, the research of data-scarce recognition from skeleton sequences, such as one-shot action recognition, does not explicitly consider occlusions despite their everyday pervasiveness.
Ranked #1 on
Action Classification
on Toyota Smarthome dataset
(Accuracy metric)
CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving
In this paper, we propose a new deep network architecture for optical flow estimation in autonomous driving--CSFlow, which consists of two novel modules: Cross Strip Correlation module (CSC) and Correlation Regression Initialization module (CRI).
Should I take a walk? Estimating Energy Expenditure from Video Data
To study this underresearched task, we introduce Vid2Burn -- an omni-source benchmark for estimating caloric expenditure from video data featuring both, high- and low-intensity activities for which we derive energy expenditure annotations based on models established in medical literature.
Exploring Event-driven Dynamic Context for Accident Scene Segmentation
Moreover, in order to evaluate the segmentation performance in traffic accidents, we provide a pixel-wise annotated accident dataset, namely DADA-seg, which contains a variety of critical scenarios from traffic accidents.
Ranked #3 on
Semantic Segmentation
on DADA-seg
(using extra training data)
Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning
Human affect recognition is a well-established research area with numerous applications, e. g., in psychological care, but existing methods assume that all emotions-of-interest are given a priori as annotated training examples.
Event-Based Fusion for Motion Deblurring with Cross-modal Attention
Traditional frame-based cameras inevitably suffer from motion blur due to long exposure times.
Ranked #3 on
Deblurring
on GoPro
(using extra training data)
Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
We look at this problem from the perspective of domain adaptation and bring panoramic semantic segmentation to a setting, where labelled training data originates from a different distribution of conventional pinhole camera images.
Ranked #7 on
Semantic Segmentation
on DensePASS
(using extra training data)
Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
In this paper, we build a wearable system with a novel dual-head Transformer for Transparency (Trans4Trans) perception model, which can segment general- and transparent objects.
Ranked #2 on
Semantic Segmentation
on DADA-seg
(using extra training data)
Panoramic Depth Estimation via Supervised and Unsupervised Learning in Indoor Scenes
With a comprehensive variety of experiments, this research demonstrates the effectiveness of our schemes aiming for indoor scene perception.
Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation
By recognizing the color of pedestrian traffic lights, our prototype can help the user to cross a street safely.
DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
First, we formalize the task of unsupervised domain adaptation for panoramic semantic segmentation, where a network trained on labelled examples from the source domain of pinhole camera data is deployed in a different target domain of panoramic images, for which no labels are available.
Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World
Common fully glazed facades and transparent objects present architectural barriers and impede the mobility of people with low vision or blindness, for instance, a path detected behind a glass door is inaccessible unless it is correctly perceived and reacted.
Ranked #1 on
Semantic Segmentation
on Trans10K
HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor
To tackle these issues, we propose HIDA, a lightweight assistive system based on 3D point cloud instance segmentation with a solid-state LiDAR sensor, for holistic indoor detection and avoidance.
Ranked #17 on
3D Instance Segmentation
on ScanNet(v2)
MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding
At the heart of all automated driving systems is the ability to sense the surroundings, e. g., through semantic segmentation of LiDAR sequences, which experienced a remarkable progress due to the release of large datasets such as SemanticKITTI and nuScenes-LidarSeg.
Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos
A lightweight panoramic annular semantic segmentation neural network model is designed to achieve high-accuracy and real-time scene parsing.
Ranked #69 on
Semantic Segmentation
on Cityscapes val
Capturing Omni-Range Context for Omnidirectional Segmentation
Convolutional Networks (ConvNets) excel at semantic segmentation and have become a vital component for perception in autonomous driving.
Ranked #10 on
Semantic Segmentation
on DensePASS
(using extra training data)
Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired
Based on Lintention, we then devise a novel panoptic segmentation model which we term Panoptic Lintention Net.
Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired
As the scene information, including objectness and scene type, are important for people with visual impairment, in this work we present a multi-task efficient perception system for the scene parsing and recognition tasks.
Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning
In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.
DR-TANet: Dynamic Receptive Temporal Attention Network for Street Scene Change Detection
Street scene change detection continues to capture researchers' interests in the computer vision community.
Panoramic annular SLAM with loop closure and global optimization
In this paper, we propose panoramic annular simultaneous localization and mapping (PA-SLAM), a visual SLAM system based on panoramic annular lens.
Loop Closure Detection
Simultaneous Localization and Mapping
Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusion
Semantic Segmentation (SS) is promising for outdoor scene perception in safety-critical applications like autonomous vehicles, assisted navigation and so on.
Ranked #5 on
Semantic Segmentation
on UPLight
ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data
Ensuring the safety of all traffic participants is a prerequisite for bringing intelligent vehicles closer to practical applications.
Ranked #6 on
Semantic Segmentation
on KITTI-360
Can we cover navigational perception needs of the visually impaired by panoptic segmentation?
Navigational perception for visually impaired people has been substantially promoted by both classic and deep learning based segmentation methods.
Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images
Semantic segmentation has made striking progress due to the success of deep convolutional neural networks.
Ranked #11 on
Semantic Segmentation
on EventScape
Universal Semantic Segmentation for Fisheye Urban Driving Images
In this paper, a seven degrees of freedom (DoF) augmentation method is proposed to transform rectilinear image to fisheye image in a more comprehensive way.
DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensing
Semantically interpreting the traffic scene is crucial for autonomous transportation and robotics systems.
Ranked #35 on
Semantic Segmentation
on DensePASS
Semantic Segmentation of Panoramic Images Using a Synthetic Dataset
In addition, it has been shown that by using panoramic images with a 180 degree FoV as training data the model has better performance.
See Clearer at Night: Towards Robust Nighttime Semantic Segmentation through Day-Night Image Conversion
However, in face of adverse conditions such as the nighttime, semantic segmentation loses its accuracy significantly.
Ranked #8 on
Semantic Segmentation
on Nighttime Driving
A Multimodal Vision Sensor for Autonomous Driving
This paper describes a multimodal vision sensor that integrates three types of cameras, including a stereo camera, a polarization camera and a panoramic camera.
A Comparative Study of High-Recall Real-Time Semantic Segmentation Based on Swift Factorized Network
Furthermore, we make a detailed analysis and comparison of the three proposed methods on the promotion of recall rate.
Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems
Semantic Segmentation (SS) is a task to assign semantic label to each pixel of the images, which is of immense significance for autonomous vehicles, robotics and assisted navigation of vulnerable road users.
ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
The main contributions lie in the Attention Complementary Module (ACM) and the architecture with three parallel branches.
Ranked #4 on
Semantic Segmentation
on KITTI-360
Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors
The panoramic annular images captured by the single camera are processed and fed into the NetVLAD network to form the active deep descriptor, and sequential matching is utilized to generate the localization result.
Visual Localization of Key Positions for Visually Impaired People
On the off-the-shelf navigational assistance devices, the localization precision is limited to the signal error of global navigation satellite system (GNSS).
