Search Results for author: Kailun Yang

Found 96 papers, 78 papers with code

DTCLMapper: Dual Temporal Consistent Learning for Vectorized HD Map Construction

no code implementations9 May 2024 Siyu Li, Jiacheng Lin, Hao Shi, Jiaming Zhang, Song Wang, You Yao, Zhiyong Li, Kailun Yang

In this paper, we revisit the temporal fusion of vectorized HD maps, focusing on temporal instance consistency and temporal map consistency learning.

Contrastive Learning Scene Understanding +1

Design, analysis, and manufacturing of a glass-plastic hybrid minimalist aspheric panoramic annular lens

no code implementations5 May 2024 Shaohua Gao, Qi Jiang, Yiqi Liao, Yi Qiu, Wanglei Ying, Kailun Yang, Kaiwei Wang, Benhao Zhang, Jian Bai

We propose a high-performance glass-plastic hybrid minimalist aspheric panoramic annular lens (ASPAL) to solve several major limitations of the traditional panoramic annular lens (PAL), such as large size, high weight, and complex system.

Towards Consistent Object Detection via LiDAR-Camera Synergy

1 code implementation2 May 2024 Kai Luo, Hao Wu, Kefu Yi, Kailun Yang, Wei Hao, Rongdong Hu

In light of this, this paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework that requires only a single forward inference to simultaneously obtain an object's position in both point clouds and images and establish their correlation.

Object object-detection +2

Global Search Optics: Automatically Exploring Optimal Solutions to Compact Computational Imaging Systems

no code implementations30 Apr 2024 Yao Gao, Qi Jiang, Shaohua Gao, Lei Sun, Kailun Yang, Kaiwei Wang

In this work, we present Global Search Optics (GSO) to automatically design compact computational imaging systems through two parts: (i) Fused Optimization Method for Automatic Optical Design (OptiFusion), which searches for diverse initial optical systems under certain design specifications; and (ii) Efficient Physic-aware Joint Optimization (EPJO), which conducts parallel joint optimization of initial optical systems and image reconstruction networks with the consideration of physical constraints, culminating in the selection of the optimal solution.

Image Reconstruction

CFMW: Cross-modality Fusion Mamba for Multispectral Object Detection under Adverse Weather Conditions

1 code implementation25 Apr 2024 Haoyuan Li, Qi Hu, You Yao, Kailun Yang, Peng Chen

Furthermore, we introduce the Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment detection accuracy in adverse weather conditions.

Multispectral Object Detection Object +2

MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model

1 code implementation19 Apr 2024 Kang Zeng, Hao Shi, Jiacheng Lin, Siyu Li, Jintao Cheng, Kaiwei Wang, Zhiyong Li, Kailun Yang

In this paper, we propose a novel LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model, termed MambaMOS.

Object Semantic Segmentation

Real-World Computational Aberration Correction via Quantized Domain-Mixing Representation

1 code implementation15 Mar 2024 Qi Jiang, Zhonghua Yi, Shaohua Gao, Yao Gao, Xiaolong Qian, Hao Shi, Lei Sun, Zhijie Xu, Kailun Yang, Kaiwei Wang

Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications.

Unsupervised Domain Adaptation

Skeleton-Based Human Action Recognition with Noisy Labels

1 code implementation15 Mar 2024 Yi Xu, Kunyu Peng, Di Wen, Ruiping Liu, Junwei Zheng, Yufan Chen, Jiaming Zhang, Alina Roitberg, Kailun Yang, Rainer Stiefelhagen

In this study, we bridge this gap by implementing a framework that augments well-established skeleton-based human action recognition methods with label-denoising strategies from various research areas to serve as the initial benchmark.

Action Recognition Denoising +3

OccFiner: Offboard Occupancy Refinement with Hybrid Propagation

no code implementations13 Mar 2024 Hao Shi, Song Wang, Jiaming Zhang, Xiaoting Yin, Zhongdao Wang, Zhijian Zhao, Guangming Wang, Jianke Zhu, Kailun Yang, Kaiwei Wang

Vision-based occupancy prediction, also known as 3D Semantic Scene Completion (SSC), presents a significant challenge in computer vision.

3D Semantic Scene Completion

EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving

1 code implementation28 Feb 2024 Jiacheng Lin, Jiajun Chen, Kunyu Peng, Xuan He, Zhiyong Li, Rainer Stiefelhagen, Kailun Yang

This paper introduces the task of Auditory Referring Multi-Object Tracking (AR-MOT), which dynamically tracks specific objects in a video sequence based on audio expressions and appears as a challenging problem in autonomous driving.

Autonomous Driving Multi-Object Tracking +1

Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation

1 code implementation30 Jan 2024 Ruiping Liu, Jiaming Zhang, Kunyu Peng, Yufan Chen, Ke Cao, Junwei Zheng, M. Saquib Sarfraz, Kailun Yang, Rainer Stiefelhagen

Integrating information from multiple modalities enhances the robustness of scene perception systems in autonomous vehicles, providing a more comprehensive and reliable sensory framework.

Autonomous Vehicles Scene Segmentation

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

1 code implementation30 Jan 2024 Jianbin Jiao, Xina Cheng, WeiJie Chen, Xiaoting Yin, Hao Shi, Kailun Yang

Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content.

3D Human Pose Estimation Scene Understanding

LF Tracy: A Unified Single-Pipeline Approach for Salient Object Detection in Light Field Cameras

1 code implementation30 Jan 2024 Fei Teng, Jiaming Zhang, Jiawei Liu, Kunyu Peng, Xina Cheng, Zhiyong Li, Kailun Yang

Previous approaches predominantly employ a custom two-stream design to discover the implicit angular feature within light field cameras, leading to significant information isolation between different LF representations.

Data Augmentation Decoder +3

Navigating Open Set Scenarios for Skeleton-based Action Recognition

1 code implementation11 Dec 2023 Kunyu Peng, Cheng Yin, Junwei Zheng, Ruiping Liu, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg

In real-world scenarios, human actions often fall outside the distribution of training data, making it crucial for models to recognize known actions and reject unknown ones.

Novelty Detection Open Set Action Recognition +3

Rethinking Event-based Human Pose Estimation with 3D Event Representations

1 code implementation8 Nov 2023 Xiaoting Yin, Hao Shi, Jiaan Chen, Ze Wang, Yaozu Ye, Huajian Ni, Kailun Yang, Kaiwei Wang

Experiments on EV-3DPW demonstrate that the robustness of our proposed 3D representation methods compared to traditional RGB images and event frame techniques under the same backbones.

Autonomous Driving Pose Estimation

CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity

1 code implementation4 Oct 2023 Hao Shi, Chengshan Pang, Jiaming Zhang, Kailun Yang, Yuhao Wu, Huajian Ni, Yining Lin, Rainer Stiefelhagen, Kaiwei Wang

Roadside camera-driven 3D object detection is a crucial task in intelligent transportation systems, which extends the perception range beyond the limitations of vision-centric vehicles and enhances road safety.

feature selection Monocular 3D Object Detection +1

Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision

1 code implementation21 Sep 2023 Yiping Wei, Kunyu Peng, Alina Roitberg, Jiaming Zhang, Junwei Zheng, Ruiping Liu, Yufan Chen, Kailun Yang, Rainer Stiefelhagen

These works overlooked the differences in performance among modalities, which led to the propagation of erroneous knowledge between modalities while only three fundamental modalities, i. e., joints, bones, and motions are used, hence no additional modalities are explored.

Action Recognition Knowledge Distillation +3

S$^3$-MonoDETR: Supervised Shape&Scale-perceptive Deformable Transformer for Monocular 3D Object Detection

no code implementations2 Sep 2023 Xuan He, Kailun Yang, Junwei Zheng, Jin Yuan, Luis M. Bergasa, HUI ZHANG, Zhiyong Li

These methods typically use visual and depth representations to generate query points on objects, whose quality plays a decisive role in the detection accuracy.

Monocular 3D Object Detection object-detection

FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

1 code implementation14 Aug 2023 Zhonghua Yi, Hao Shi, Kailun Yang, Qi Jiang, Yaozu Ye, Ze Wang, Huajian Ni, Kaiwei Wang

Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point Control Loss (CPCL) function for diverse point-wise supervision; 2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition Control Encoder (CCE).

Autonomous Driving Optical Flow Estimation +1

Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation

1 code implementation2 Aug 2023 Guojin Zhong, Jin Yuan, Pan Wang, Kailun Yang, Weili Guan, Zhiyong Li

The recently rising markup-to-image generation poses greater challenges as compared to natural image generation, due to its low tolerance for errors as well as the complex sequence and context correlations between markup and rendered image.

Denoising Image Generation

OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation

2 code implementations28 Jul 2023 Fei Teng, Jiaming Zhang, Kunyu Peng, Yaonan Wang, Rainer Stiefelhagen, Kailun Yang

To avoid feature loss during network propagation and simultaneously streamline the redundant information from the light field camera, we present a simple yet very effective Sub-Aperture Fusion Module (SAFM) to embed sub-aperture images into angular features without any additional memory cost.

Autonomous Driving Scene Understanding +1

Tightly-Coupled LiDAR-Visual SLAM Based on Geometric Features for Mobile Agents

no code implementations15 Jul 2023 Ke Cao, Ruiping Liu, Ze Wang, Kunyu Peng, Jiaming Zhang, Junwei Zheng, Zhifeng Teng, Kailun Yang, Rainer Stiefelhagen

On the other hand, the entire line segment detected by the visual subsystem overcomes the limitation of the LiDAR subsystem, which can only perform the local calculation for geometric features.

Autonomous Navigation Pose Estimation +2

Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairments

1 code implementation15 Jul 2023 Ruiping Liu, Jiaming Zhang, Kunyu Peng, Junwei Zheng, Ke Cao, Yufan Chen, Kailun Yang, Rainer Stiefelhagen

Grounded Situation Recognition (GSR) is capable of recognizing and interpreting visual scenes in a contextually intuitive way, yielding salient activities (verbs) and the involved entities (roles) depicted in images.

Decoder Grounded Situation Recognition +2

Towards Anytime Optical Flow Estimation with Event Cameras

1 code implementation11 Jul 2023 Yaozu Ye, Hao Shi, Kailun Yang, Ze Wang, Xiaoting Yin, Yining Lin, Mao Liu, Yaonan Wang, Kaiwei Wang

We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision.

Autonomous Driving Motion Estimation +1

Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers

1 code implementation22 Jun 2023 Qi Jiang, Shaohua Gao, Yao Gao, Kailun Yang, Zhonghua Yi, Hao Shi, Lei Sun, Kaiwei Wang

In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to address minimalist and high-quality panoramic imaging.


LF-PGVIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras using Points and Geodesic Segments

1 code implementation11 Jun 2023 Ze Wang, Kailun Yang, Hao Shi, Yufan Zhang, Zhijie Xu, Fei Gao, Kaiwei Wang

The purpose of our research is to unleash the potential of point-line odometry with large-FoV omnidirectional cameras, even for cameras with negative-plane FoV.

Line Detection

VPUFormer: Visual Prompt Unified Transformer for Interactive Image Segmentation

1 code implementation11 Jun 2023 Xu Zhang, Kailun Yang, Jiacheng Lin, Jin Yuan, Zhiyong Li, Shutao Li

Specifically, we design a Prompt-unified Encoder (PuE) by using Gaussian mapping to generate a unified one-dimensional vector for click, box, and scribble prompts, which well captures users' intentions as well as provides a denser representation of user prompts.

Image Segmentation Segmentation +1

Exploring Few-Shot Adaptation for Activity Recognition on Diverse Domains

2 code implementations15 May 2023 Kunyu Peng, Di Wen, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg

In this work, we focus on Few-Shot Domain Adaptation for Activity Recognition (FSDA-AR), which leverages a very small amount of labeled target videos to achieve effective adaptation.

Action Recognition Unsupervised Domain Adaptation

SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection

1 code implementation12 May 2023 Xuan He, Fan Yang, Kailun Yang, Jiacheng Lin, Haolong Fu, Meng Wang, Jin Yuan, Zhiyong Li

To tackle this problem, this paper proposes a novel "Supervised Scale-aware Deformable Attention" (SSDA) for monocular 3D object detection.

Monocular 3D Object Detection Object +1

Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving

1 code implementation7 May 2023 Siyu Li, Kailun Yang, Hao Shi, Jiaming Zhang, Jiacheng Lin, Zhifeng Teng, Zhiyong Li

At the same time, an Across-Space Loss (ASL) is designed to mitigate the negative impact of geometric distortions.

Autonomous Driving

FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation

1 code implementation24 Mar 2023 Hao Shi, Yu Li, Kailun Yang, Jiaming Zhang, Kunyu Peng, Alina Roitberg, Yaozu Ye, Huajian Ni, Kaiwei Wang, Rainer Stiefelhagen

This paper raises the new task of Fisheye Semantic Completion (FSC), where dense texture, structure, and semantics of a fisheye image are inferred even beyond the sensor field-of-view (FoV).

Image Outpainting Semantic Segmentation

PanoVPR: Towards Unified Perspective-to-Equirectangular Visual Place Recognition via Sliding Windows across the Panoramic View

1 code implementation24 Mar 2023 Ze Shi, Hao Shi, Kailun Yang, Zhe Yin, Yining Lin, Kaiwei Wang

To address this, we propose \textit{PanoVPR}, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping.

Autonomous Driving Image Retrieval +2

Towards Activated Muscle Group Estimation in the Wild

1 code implementation2 Mar 2023 Kunyu Peng, David Schneider, Alina Roitberg, Kailun Yang, Jiaming Zhang, Chen Deng, Kaiyu Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen

In this paper, we tackle the new task of video-based Activated Muscle Group Estimation (AMGE) aiming at identifying active muscle regions during physical activity in the wild.

Human Activity Recognition Knowledge Distillation +1

Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformer

3 code implementations21 Nov 2022 Hao Shi, Qi Jiang, Kailun Yang, Xiaoting Yin, Huajian Ni, Kaiwei Wang

In this paper, we propose the concept of online video inpainting for autonomous vehicles to expand the field of view, thereby enhancing scene visibility, perception, and system safety.

Autonomous Vehicles object-detection +4

Computational Imaging for Machine Perception: Transferring Semantic Segmentation beyond Aberrations

1 code implementation21 Nov 2022 Qi Jiang, Hao Shi, Shaohua Gao, Jiaming Zhang, Kailun Yang, Lei Sun, Huajian Ni, Kaiwei Wang

Further, we propose Computational Imaging Assisted Domain Adaptation (CIADA) to leverage prior knowledge of CI for robust performance in SSOA.

Scene Understanding Semantic Segmentation +1

LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents

3 code implementations12 Sep 2022 Ze Wang, Kailun Yang, Hao Shi, Peng Li, Fei Gao, Jian Bai, Kaiwei Wang

As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable.

Autonomous Driving Simultaneous Localization and Mapping

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

1 code implementation25 Jul 2022 Jiaming Zhang, Kailun Yang, Hao Shi, Simon Reiß, Kunyu Peng, Chaoxiang Ma, Haodong Fu, Philip H. S. Torr, Kaiwei Wang, Rainer Stiefelhagen

In this paper, we address panoramic semantic segmentation which is under-explored due to two critical challenges: (1) image distortions and object deformations on panoramas; (2) lack of semantic annotations in the 360-degree imagery.

Pseudo Label Segmentation +2

Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers

1 code implementation13 Jul 2022 Chang Chen, Jiaming Zhang, Kailun Yang, Kunyu Peng, Rainer Stiefelhagen

Humans have an innate ability to sense their surroundings, as they can extract the spatial representation from the egocentric perception and form an allocentric semantic map via spatial transformation and memory updating.

Decoder Semantic Segmentation

Multi-modal Depression Estimation based on Sub-attentional Fusion

1 code implementation13 Jul 2022 Ping-Cheng Wei, Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen

Failure to timely diagnose and effectively treat depression leads to over 280 million people suffering from this psychological disorder worldwide.

Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning

1 code implementation21 Jun 2022 Alexander Jaus, Kailun Yang, Rainer Stiefelhagen

In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to the panoramic domain in a cost-minimizing way.

Contrastive Learning Domain Generalization +3

Annular Computational Imaging: Capture Clear Panoramic Images through Simple Lens

1 code implementation13 Jun 2022 Qi Jiang, Hao Shi, Lei Sun, Shaohua Gao, Kailun Yang, Kaiwei Wang

In this paper, we propose an Annular Computational Imaging (ACI) framework to break the optical limit of light-weight PAL design.

Image Restoration

Efficient Human Pose Estimation via 3D Event Point Cloud

1 code implementation9 Jun 2022 Jiaan Chen, Hao Shi, Yaozu Ye, Kailun Yang, Lei Sun, Kaiwei Wang

We then leverage the rasterized event point cloud as input to three different backbones, PointNet, DGCNN, and Point Transformer, with two linear layer decoders to predict the location of human keypoints.

3D Human Pose Estimation Edge-computing +1

Review on Panoramic Imaging and Its Applications in Scene Understanding

no code implementations11 May 2022 Shaohua Gao, Kailun Yang, Hao Shi, Kaiwei Wang, Jian Bai

However, while satisfying the need for large-FoV photographic imaging, panoramic imaging instruments are expected to have high resolution, no blind area, miniaturization, and multidimensional intelligent perception, and can be combined with artificial intelligence methods towards the next generation of intelligent instruments, enabling deeper understanding and more holistic perception of 360-degree real-world surrounding environments.

Autonomous Driving Depth Estimation +4

Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates

no code implementations10 Apr 2022 Alina Roitberg, Kunyu Peng, David Schneider, Kailun Yang, Marios Koulakis, Manuel Martinez, Rainer Stiefelhagen

In this work, we for the first time examine how well the confidence values of modern driver observation models indeed match the probability of the correct outcome and show that raw neural network-based approaches tend to significantly overestimate their prediction quality.

Action Recognition Image Classification

MatchFormer: Interleaving Attention in Transformers for Feature Matching

1 code implementation17 Mar 2022 Qing Wang, Jiaming Zhang, Kailun Yang, Kunyu Peng, Rainer Stiefelhagen

While detector-based methods coupled with feature descriptors struggle in low-texture scenes, CNN-based methods with a sequential extract-to-match pipeline, fail to make use of the matching capacity of the encoder and tend to overburden the decoder for matching.

Decoder Homography Estimation +2

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers

1 code implementation9 Mar 2022 Jiaming Zhang, Huayao Liu, Kailun Yang, Xinxin Hu, Ruiping Liu, Rainer Stiefelhagen

Pixel-wise semantic segmentation of RGB images can be advanced by exploiting complementary features from the supplementary modality (X-modality).

Autonomous Vehicles Image Segmentation +5

TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

1 code implementation2 Mar 2022 Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen

This module operates in the latent feature-space enriching and diversifying the training set at feature-level in order to improve generalization to novel data appearances, (e. g., sensor changes) and general feature quality.

Human Activity Recognition

Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

1 code implementation CVPR 2022 Jiaming Zhang, Kailun Yang, Chaoxiang Ma, Simon Reiß, Kunyu Peng, Rainer Stiefelhagen

To get around this domain difference and bring together semantic annotations from pinhole- and 360-degree surround-visuals, we propose to learn object deformations and panoramic image distortions in the Deformable Patch Embedding (DPE) and Deformable MLP (DMLP) components which blend into our Transformer for PAnoramic Semantic Segmentation (Trans4PASS) model.

Scene Understanding Semantic Segmentation +1

TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

2 code implementations27 Feb 2022 Ruiping Liu, Kailun Yang, Alina Roitberg, Jiaming Zhang, Kunyu Peng, Huayao Liu, Yaonan Wang, Rainer Stiefelhagen

Semantic segmentation benchmarks in the realm of autonomous driving are dominated by large pre-trained transformers, yet their widespread adoption is impeded by substantial computational costs and prolonged training durations.

Autonomous Driving Knowledge Distillation +3

Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions

2 code implementations23 Feb 2022 Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen

Yet, the research of data-scarce recognition from skeleton sequences, such as one-shot action recognition, does not explicitly consider occlusions despite their everyday pervasiveness.

Action Classification Action Recognition +2

CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving

1 code implementation2 Feb 2022 Hao Shi, Yifan Zhou, Kailun Yang, Xiaoting Yin, Kaiwei Wang

In this paper, we propose a new deep network architecture for optical flow estimation in autonomous driving--CSFlow, which consists of two novel modules: Cross Strip Correlation module (CSC) and Correlation Regression Initialization module (CRI).

Autonomous Driving Optical Flow Estimation

Should I take a walk? Estimating Energy Expenditure from Video Data

1 code implementation1 Feb 2022 Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen

To study this underresearched task, we introduce Vid2Burn -- an omni-source benchmark for estimating caloric expenditure from video data featuring both, high- and low-intensity activities for which we derive energy expenditure annotations based on models established in medical literature.

Video Recognition

Exploring Event-driven Dynamic Context for Accident Scene Segmentation

1 code implementation9 Dec 2021 Jiaming Zhang, Kailun Yang, Rainer Stiefelhagen

Moreover, in order to evaluate the segmentation performance in traffic accidents, we provide a pixel-wise annotated accident dataset, namely DADA-seg, which contains a variety of critical scenarios from traffic accidents.

Ranked #3 on Semantic Segmentation on DADA-seg (using extra training data)

Scene Segmentation Segmentation

Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning

1 code implementation30 Nov 2021 Kunyu Peng, Alina Roitberg, David Schneider, Marios Koulakis, Kailun Yang, Rainer Stiefelhagen

Human affect recognition is a well-established research area with numerous applications, e. g., in psychological care, but existing methods assume that all emotions-of-interest are given a priori as annotated training examples.

Emotion Recognition Metric Learning +1

Event-Based Fusion for Motion Deblurring with Cross-modal Attention

1 code implementation30 Nov 2021 Lei Sun, Christos Sakaridis, Jingyun Liang, Qi Jiang, Kailun Yang, Peng Sun, Yaozu Ye, Kaiwei Wang, Luc van Gool

Traditional frame-based cameras inevitably suffer from motion blur due to long exposure times.

Ranked #3 on Deblurring on GoPro (using extra training data)

Deblurring Image Deblurring +1

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation

1 code implementation21 Oct 2021 Jiaming Zhang, Chaoxiang Ma, Kailun Yang, Alina Roitberg, Kunyu Peng, Rainer Stiefelhagen

We look at this problem from the perspective of domain adaptation and bring panoramic semantic segmentation to a setting, where labelled training data originates from a different distribution of conventional pinhole camera images.

Ranked #7 on Semantic Segmentation on DensePASS (using extra training data)

Autonomous Vehicles Segmentation +2

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

1 code implementation20 Aug 2021 Jiaming Zhang, Kailun Yang, Angela Constantinescu, Kunyu Peng, Karin Müller, Rainer Stiefelhagen

In this paper, we build a wearable system with a novel dual-head Transformer for Transparency (Trans4Trans) perception model, which can segment general- and transparent objects.

Ranked #2 on Semantic Segmentation on DADA-seg (using extra training data)

Decoder Navigate +2

Panoramic Depth Estimation via Supervised and Unsupervised Learning in Indoor Scenes

1 code implementation18 Aug 2021 Keyang Zhou, Kailun Yang, Kaiwei Wang

With a comprehensive variety of experiments, this research demonstrates the effectiveness of our schemes aiming for indoor scene perception.

Camera Calibration Monocular Depth Estimation +2

DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange

1 code implementation13 Aug 2021 Chaoxiang Ma, Jiaming Zhang, Kailun Yang, Alina Roitberg, Rainer Stiefelhagen

First, we formalize the task of unsupervised domain adaptation for panoramic semantic segmentation, where a network trained on labelled examples from the source domain of pinhole camera data is deployed in a different target domain of panoramic images, for which no labels are available.

Segmentation Semantic Segmentation +1

Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World

1 code implementation7 Jul 2021 Jiaming Zhang, Kailun Yang, Angela Constantinescu, Kunyu Peng, Karin Müller, Rainer Stiefelhagen

Common fully glazed facades and transparent objects present architectural barriers and impede the mobility of people with low vision or blindness, for instance, a path detected behind a glass door is inaccessible unless it is correctly perceived and reacted.

Decoder Navigate +2

MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding

1 code implementation1 Jul 2021 Kunyu Peng, Juncong Fei, Kailun Yang, Alina Roitberg, Jiaming Zhang, Frank Bieder, Philipp Heidenreich, Christoph Stiller, Rainer Stiefelhagen

At the heart of all automated driving systems is the ability to sense the surroundings, e. g., through semantic segmentation of LiDAR sequences, which experienced a remarkable progress due to the release of large datasets such as SemanticKITTI and nuScenes-LidarSeg.

3D Object Detection Graph Attention +4

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

no code implementations15 May 2021 Lei Sun, Jia Wang, Kailun Yang, Kaikai Wu, Xiangdong Zhou, Kaiwei Wang, Jian Bai

A lightweight panoramic annular semantic segmentation neural network model is designed to achieve high-accuracy and real-time scene parsing.

Scene Parsing Scene Segmentation +1

Capturing Omni-Range Context for Omnidirectional Segmentation

1 code implementation CVPR 2021 Kailun Yang, Jiaming Zhang, Simon Reiß, Xinxin Hu, Rainer Stiefelhagen

Convolutional Networks (ConvNets) excel at semantic segmentation and have become a vital component for perception in autonomous driving.

Ranked #10 on Semantic Segmentation on DensePASS (using extra training data)

Autonomous Driving Image Segmentation +2

Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired

no code implementations6 Mar 2021 Yingzhi Zhang, Haoye Chen, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen

As the scene information, including objectness and scene type, are important for people with visual impairment, in this work we present a multi-task efficient perception system for the scene parsing and recognition tasks.

Real-Time Semantic Segmentation Scene Recognition

Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning

1 code implementation1 Mar 2021 Alexander Jaus, Kailun Yang, Rainer Stiefelhagen

In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.

Contrastive Learning Panoptic Segmentation +2

Panoramic annular SLAM with loop closure and global optimization

no code implementations26 Feb 2021 Hao Chen, Weijian Hu, Kailun Yang, Jian Bai, Kaiwei Wang

In this paper, we propose panoramic annular simultaneous localization and mapping (PA-SLAM), a visual SLAM system based on panoramic annular lens.

Loop Closure Detection Simultaneous Localization and Mapping

Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusion

1 code implementation26 Nov 2020 Kaite Xiang, Kailun Yang, Kaiwei Wang

Semantic Segmentation (SS) is promising for outdoor scene perception in safety-critical applications like autonomous vehicles, assisted navigation and so on.

Autonomous Vehicles Semantic Segmentation

ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data

1 code implementation20 Aug 2020 Jiaming Zhang, Kailun Yang, Rainer Stiefelhagen

Ensuring the safety of all traffic participants is a prerequisite for bringing intelligent vehicles closer to practical applications.

Autonomous Vehicles Benchmarking +2

Can we cover navigational perception needs of the visually impaired by panoptic segmentation?

no code implementations20 Jul 2020 Wei Mao, Jiaming Zhang, Kailun Yang, Rainer Stiefelhagen

Navigational perception for visually impaired people has been substantially promoted by both classic and deep learning based segmentation methods.

Instance Segmentation Panoptic Segmentation +1

Universal Semantic Segmentation for Fisheye Urban Driving Images

1 code implementation31 Jan 2020 Yaozu Ye, Kailun Yang, Kaite Xiang, Juan Wang, Kaiwei Wang

In this paper, a seven degrees of freedom (DoF) augmentation method is proposed to transform rectilinear image to fisheye image in a more comprehensive way.

Autonomous Driving Image Segmentation +2

Semantic Segmentation of Panoramic Images Using a Synthetic Dataset

1 code implementation2 Sep 2019 Yuanyou Xu, Kaiwei Wang, Kailun Yang, Dongming Sun, Jia Fu

In addition, it has been shown that by using panoramic images with a 180 degree FoV as training data the model has better performance.

Segmentation Semantic Segmentation

A Multimodal Vision Sensor for Autonomous Driving

no code implementations15 Aug 2019 Dongming Sun, Xiao Huang, Kailun Yang

This paper describes a multimodal vision sensor that integrates three types of cameras, including a stereo camera, a polarization camera and a panoramic camera.

Autonomous Driving Semantic Segmentation

Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems

1 code implementation25 Jul 2019 Kaite Xiang, Kaiwei Wang, Kailun Yang

Semantic Segmentation (SS) is a task to assign semantic label to each pixel of the images, which is of immense significance for autonomous vehicles, robotics and assisted navigation of vulnerable road users.

Autonomous Vehicles Segmentation +1

Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors

2 code implementations14 May 2019 Ruiqi Cheng, Kaiwei Wang, Shufei Lin, Weijian Hu, Kailun Yang, Xiao Huang, Huabing Li, Dongming Sun, Jian Bai

The panoramic annular images captured by the single camera are processed and fed into the NetVLAD network to form the active deep descriptor, and sequential matching is utilized to generate the localization result.

Autonomous Vehicles Camera Localization +2

Visual Localization of Key Positions for Visually Impaired People

no code implementations9 Oct 2018 Ruiqi Cheng, Kaiwei Wang, Longqing Lin, Kailun Yang

On the off-the-shelf navigational assistance devices, the localization precision is limited to the signal error of global navigation satellite system (GNSS).

Visual Localization

Cannot find the paper you are looking for? You can Submit a new open access paper.