Search Results for author: Alois Knoll

Found 145 papers, 59 papers with code

Optimizing Retrieval Augmented Generation for Object Constraint Language

no code implementations19 May 2025 Kevin Chenhao Li, Vahid Zolfaghari, Nenad Petrovic, Fengjunjie Pan, Alois Knoll

Our experimental results, focusing on retrieval-augmented generation, indicate that while retrieval can enhance generation accuracy, its effectiveness depends on the retrieval method and the number of retrieved chunks (k).

Object RAG +3

E2E Parking Dataset: An Open Benchmark for End-to-End Autonomous Parking

no code implementations15 Apr 2025 Kejia Gao, Liguo Zhou, Mingjun Liu, Alois Knoll

End-to-end learning has shown great potential in autonomous parking, yet the lack of publicly available datasets limits reproducibility and benchmarking.

Benchmarking Position

BiSeg-SAM: Weakly-Supervised Post-Processing Framework for Boosting Binary Segmentation in Segment Anything Models

no code implementations2 Apr 2025 Encheng Su, Hu Cao, Alois Knoll

While various segmentation methods for polyps and skin lesions using fully supervised deep learning techniques have been developed, the pixel-level annotation of medical images by doctors is both time-consuming and costly.

Segmentation

Data-Agnostic Robotic Long-Horizon Manipulation with Vision-Language-Guided Closed-Loop Feedback

1 code implementation27 Mar 2025 Yuan Meng, Xiangtong Yao, Haihui Ye, Yirui Zhou, Shengqiang Zhang, Zhenshan Bing, Alois Knoll

Recent advances in language-conditioned robotic manipulation have leveraged imitation and reinforcement learning to enable robots to execute tasks from human commands.

Task Planning

Pretrained Bayesian Non-parametric Knowledge Prior in Robotic Long-Horizon Reinforcement Learning

1 code implementation27 Mar 2025 Yuan Meng, Xiangtong Yao, KeJia Chen, Yansong Wu, Liding Zhang, Zhenshan Bing, Alois Knoll

Reinforcement learning (RL) methods typically learn new tasks from scratch, often disregarding prior knowledge that could accelerate the learning process.

Reinforcement Learning (RL)

Threshold Adaptation in Spiking Networks Enables Shortest Path Finding and Place Disambiguation

1 code implementation22 Mar 2025 Robin Dietrich, Tobias Fischer, Nicolai Waniek, Nico Reeb, Michael Milford, Alois Knoll, Adam D. Hines

We further present an ambiguity dependent threshold adaptation (ADTA) for identifying places in an environment with less ambiguity, enhancing the localization estimate of an agent.

Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Robotics

no code implementations13 Mar 2025 Josip Josifovski, Shangding Gu, Mohammadhossein Malmir, Haoliang Huang, Sayantan Auddy, Nicolás Navarro-Guerrero, Costas Spanos, Alois Knoll

In addition, the policies pretrained in the domain-randomized simulation are fixed after deployment due to the inherent instability of the optimization processes based on RL and the necessity of sampling exploitative but potentially unsafe actions on the real system.

Continual Learning Domain Adaptation +1

CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving

no code implementations9 Mar 2025 Rui Song, Chenwei Liang, Yan Xia, Walter Zimmer, Hu Cao, Holger Caesar, Andreas Festag, Alois Knoll

By aggregating and encoding both semantic and temporal deformation features, each Gaussian is equipped with cues for potential deformation compensation within 3D space, facilitating a more precise representation of dynamic scenes.

2D Semantic Segmentation 4D reconstruction +3

LLM-based Iterative Approach to Metamodeling in Automotive

no code implementations7 Mar 2025 Nenad Petrovic, Fengjunjie Pan, Vahid Zolfaghari, Alois Knoll

In this paper, we introduce an automated approach to domain-specific metamodel construction relying on Large Language Model (LLM).

Language Modeling Language Modelling +1

Multi-modal Summarization in Model-Based Engineering: Automotive Software Development Case Study

no code implementations6 Mar 2025 Nenad Petrovic, Yurui Zhang, Moaad Maaroufi, Kuo-Yi Chao, Lukasz Mazur, Fengjunjie Pan, Vahid Zolfaghari, Alois Knoll

UML and EMF diagrams in model-based engineering contain a large amount of multimodal information and intricate relational data.

LensDFF: Language-enhanced Sparse Feature Distillation for Efficient Few-Shot Dexterous Manipulation

no code implementations5 Mar 2025 Qian Feng, David S. Martinez Lema, Jianxiang Feng, Zhaopeng Chen, Alois Knoll

Learning dexterous manipulation from few-shot demonstrations is a significant yet challenging problem for advanced, human-like robotic systems.

NeRF Neural Rendering

Benchmarking Vision Foundation Models for Input Monitoring in Autonomous Driving

no code implementations14 Jan 2025 Mert Keser, Halil Ibrahim Orhan, Niki Amini-Naieni, Gesina Schwalbe, Alois Knoll, Matthias Rottmann

Current approaches for OOD classification are untested for complex domains like AD, are limited in the kinds of shifts they detect, or even require supervision with OOD samples.

Autonomous Driving Benchmarking

Learning Monocular Depth from Events via Egomotion Compensation

no code implementations26 Dec 2024 Haitao Meng, Chonghao Zhong, Sheng Tang, Lian JunJia, Wenwei Lin, Zhenshan Bing, Yi Chang, Gang Chen, Alois Knoll

To achieve this, we propose a Focus Cost Discrimination (FCD) module that measures the clarity of edges as an essential indicator of focus level and integrates spatial surroundings to facilitate cost estimation.

Monocular Depth Estimation Motion Compensation

Adopting RAG for LLM-Aided Future Vehicle Design

no code implementations14 Nov 2024 Vahid Zolfaghari, Nenad Petrovic, Fengjunjie Pan, Krzysztof Lebioda, Alois Knoll

In this paper, we explore the integration of Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) to enhance automated design and software development in the automotive industry.

Chatbot RAG +2

FuzzRisk: Online Collision Risk Estimation for Autonomous Vehicles based on Depth-Aware Object Detection via Fuzzy Inference

no code implementations9 Nov 2024 Brian Hsuan-Cheng Liao, YingJie Xu, Chih-Hong Cheng, Hasan Esen, Alois Knoll

This paper presents a novel monitoring framework that infers the level of collision risk for autonomous vehicles (AVs) based on their object detection performance.

Autonomous Vehicles Object +2

Unveiling Ontological Commitment in Multi-Modal Foundation Models

no code implementations25 Sep 2024 Mert Keser, Gesina Schwalbe, Niki Amini-Naieni, Matthias Rottmann, Alois Knoll

As a next step towards QR for validation and verification of DNNs: Concretely, we propose a method that extracts the learned superclass hierarchy from a multimodal DNN for a given set of leaf concepts.

Semantic Similarity Semantic Textual Similarity

How Could Generative AI Support Compliance with the EU AI Act? A Review for Safe Automated Driving Perception

no code implementations30 Aug 2024 Mert Keser, Youssef Shoeb, Alois Knoll

While generative AI models show promise in addressing some of the EU AI Acts requirements, such as transparency and robustness, this review examines their potential benefits and discusses how developers could leverage these methods to enhance compliance with the Act.

Autonomous Driving

Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection

no code implementations28 Aug 2024 Sondos Mohamed, Walter Zimmer, Ross Greer, Ahmed Alaaeldin Ghita, Modesto Castrillón-Santana, Mohan Trivedi, Alois Knoll, Salvatore Mario Carta, Mirko Marras

Accurately detecting 3D objects from monocular images in dynamic roadside scenarios remains a challenging problem due to varying camera perspectives and unpredictable scene conditions.

Monocular 3D Object Detection object-detection +1

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

2 code implementations19 Aug 2024 Ruiqi Zhang, Jing Hou, Florian Walter, Shangding Gu, Jiayi Guan, Florian Röhrbein, Yali Du, Panpan Cai, Guang Chen, Alois Knoll

Reinforcement Learning (RL) is a potent tool for sequential decision-making and has achieved performance surpassing human capabilities across many challenging real-world tasks.

Autonomous Driving Decision Making +6

FFHFlow: A Flow-based Variational Approach for Learning Diverse Dexterous Grasps with Shape-Aware Introspection

no code implementations21 Jul 2024 Qian Feng, Jianxiang Feng, Zhaopeng Chen, Rudolph Triebel, Alois Knoll

We further achieve performance gain by fusing this information with a discriminative grasp evaluator, facilitating a novel hybrid way for grasp evaluation.

HGL: Hierarchical Geometry Learning for Test-time Adaptation in 3D Point Cloud Segmentation

1 code implementation17 Jul 2024 Tianpei Zou, Sanqing Qu, Zhijun Li, Alois Knoll, Lianghua He, Guang Chen, Changjun Jiang

HGL comprises three complementary modules from local, global to temporal learning in a bottom-up manner. Technically, we first construct a local geometry learning module for pseudo-label generation.

Point Cloud Segmentation Pseudo Label +1

Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detection

1 code implementation17 Jul 2024 Hu Cao, Zehua Zhang, Yan Xia, Xinyi Li, Jiahao Xia, Guang Chen, Alois Knoll

The core concept is the design of the coarse-to-fine fusion module, denoted as the cross-modality adaptive feature refinement (CAFR) module.

object-detection Object Detection

Situation Monitor: Diversity-Driven Zero-Shot Out-of-Distribution Detection using Budding Ensemble Architecture for Object Detection

no code implementations5 Jun 2024 Qutub Syed, Michael Paulitsch, Korbinian Hagn, Neslihan Kose Cihangir, Kay-Ulrich Scholl, Fabian Oboril, Gereon Hinz, Alois Knoll

We introduce Situation Monitor, a novel zero-shot Out-of-Distribution (OOD) detection approach for transformer-based object detection models to enhance reliability in safety-critical machine learning applications such as autonomous driving.

Autonomous Driving Diversity +4

Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners

no code implementations1 Jun 2024 Zhi Zheng, Qian Feng, Hang Li, Alois Knoll, Jianxiang Feng

As a general-purpose reasoning machine, LLMs or Multimodal Large Language Models (MLLMs) are promising for detecting failures.

Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation

1 code implementation31 May 2024 Shangding Gu, Laixi Shi, Yuhao Ding, Alois Knoll, Costas Spanos, Adam Wierman, Ming Jin

Safe reinforcement learning (RL) is crucial for deploying RL agents in real-world applications, as it aims to maximize long-term rewards while satisfying safety constraints.

MuJoCo reinforcement-learning +3

Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning

1 code implementation26 May 2024 Shangding Gu, Bilgehan Sel, Yuhao Ding, Lu Wang, QIngwei Lin, Alois Knoll, Ming Jin

In numerous reinforcement learning (RL) problems involving safety-critical systems, a key challenge lies in balancing multiple objectives while simultaneously meeting all stringent safety constraints.

Multi-Objective Reinforcement Learning reinforcement-learning +1

GraphRelate3D: Context-Dependent 3D Object Detection with Inter-Object Relationship Graphs

no code implementations10 May 2024 MingYu Liu, Ekim Yurtsever, Marc Brede, Jun Meng, Walter Zimmer, Xingcheng Zhou, Bare Luka Zagar, Yuning Cui, Alois Knoll

In this study, we introduce an object relation module, consisting of a graph generator and a graph neural network (GNN), to learn the spatial information from certain patterns to improve 3D object detection.

3D Object Detection Autonomous Vehicles +3

Synergy of Large Language Model and Model Driven Engineering for Automated Development of Centralized Vehicular Systems

no code implementations8 Apr 2024 Nenad Petrovic, Fengjunjie Pan, Krzysztof Lebioda, Vahid Zolfaghari, Sven Kirchner, Nils Purschke, Muhammad Aqib Khan, Viktor Vorobev, Alois Knoll

We present a prototype of a tool leveraging the synergy of model driven engineering (MDE) and Large Language Models (LLM) for the purpose of software development process automation in the automotive industry.

Code Generation Language Modeling +3

Residual Chain Prediction for Autonomous Driving Path Planning

no code implementations8 Apr 2024 Liguo Zhou, Yirui Zhou, Huaming Liu, Alois Knoll

Our findings highlight the potential of Residual Chain Loss to revolutionize planning component of autonomous driving systems, marking a significant step forward in the quest for level 5 autonomous driving system.

Autonomous Driving Prediction

Omni-Kernel Network for Image Restoration

1 code implementation Proceedings of the AAAI Conference on Artificial Intelligence 2024 Yuning Cui, Wenqi Ren, Alois Knoll

Extensive experiments demonstrate that our network achieves state-of-the-art performance on 11 benchmark datasets for three representative image restoration tasks, including image dehazing, image desnowing, and image defocus deblurring.

Deblurring Image Defocus Deblurring +3

Towards Single-System Illusion in Software-Defined Vehicles -- Automated, AI-Powered Workflow

no code implementations21 Mar 2024 Krzysztof Lebioda, Viktor Vorobev, Nenad Petrovic, Fengjunjie Pan, Vahid Zolfaghari, Alois Knoll

We propose a novel model- and feature-based approach to development of vehicle software systems, where the end architecture is not explicitly defined.

AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

1 code implementation21 Mar 2024 Yuning Cui, Syed Waqas Zamir, Salman Khan, Alois Knoll, Mubarak Shah, Fahad Shahbaz Khan

Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task.

All Blind All-in-One Image Restoration +5

EC-IoU: Orienting Safety for Object Detectors via Ego-Centric Intersection-over-Union

no code implementations20 Mar 2024 Brian Hsuan-Cheng Liao, Chih-Hong Cheng, Hasan Esen, Alois Knoll

This paper presents Ego-Centric Intersection-over-Union (EC-IoU), addressing the limitation of the standard IoU measure in characterizing safety-related performance for object detectors in navigating contexts.

Autonomous Driving Object +3

TeaMs-RL: Teaching LLMs to Generate Better Instruction Datasets via Reinforcement Learning

1 code implementation13 Mar 2024 Shangding Gu, Alois Knoll, Ming Jin

The development of Large Language Models (LLMs) often confronts challenges stemming from the heavy reliance on human annotators in the reinforcement learning with human feedback (RLHF) framework, or the frequent and costly external queries tied to the self-instruct paradigm.

reinforcement-learning Reinforcement Learning +1

MAP: MAsk-Pruning for Source-Free Model Intellectual Property Protection

1 code implementation CVPR 2024 Boyang Peng, Sanqing Qu, Yong Wu, Tianpei Zou, Lianghua He, Alois Knoll, Guang Chen, Changjun Jiang

In this paper, we target a practical setting where only a well-trained source model is available and investigate how we can realize IP protection.

Dual-domain strip attention for image restoration

1 code implementation Neural Networks 2024 Yuning Cui, Alois Knoll

In this paper, we develop a dual-domain strip attention mechanism for image restoration by enhancing representation learning, which consists of spatial and frequency strip attention units.

Deblurring Image Defocus Deblurring +4

Real-Time Adaptive Safety-Critical Control with Gaussian Processes in High-Order Uncertain Models

no code implementations29 Feb 2024 Yu Zhang, long wen, Xiangtong Yao, Zhenshan Bing, Linghuan Kong, wei he, Alois Knoll

Subsequently, the hyperparameters of the Gaussian model are trained with a specially compound kernel, and the Gaussian model's online inferential capability and computational efficiency are strengthened by updating a solitary inducing point derived from new samples, in conjunction with the learned hyperparameters.

Computational Efficiency Gaussian Processes

PCDepth: Pattern-based Complementary Learning for Monocular Depth Estimation by Best of Both Worlds

no code implementations29 Feb 2024 Haotian Liu, Sanqing Qu, Fan Lu, Zongtao Bu, Florian Roehrbein, Alois Knoll, Guang Chen

Therefore, existing complementary learning approaches for MDE fuse intensity information from images and scene details from event data for better scene understanding.

Depth Prediction Monocular Depth Estimation +2

Online Efficient Safety-Critical Control for Mobile Robots in Unknown Dynamic Multi-Obstacle Environments

no code implementations26 Feb 2024 Yu Zhang, Guangyao Tian, long wen, Xiangtong Yao, Liding Zhang, Zhenshan Bing, wei he, Alois Knoll

This paper proposes a LiDAR-based goal-seeking and exploration framework, addressing the efficiency of online obstacle avoidance in unstructured environments populated with static and moving obstacles.

Path Planning based on 2D Object Bounding-box

no code implementations22 Feb 2024 Yanliang Huang, Liguo Zhou, Chang Liu, Alois Knoll

The implementation of Autonomous Driving (AD) technologies within urban environments presents significant challenges.

Autonomous Driving Graph Neural Network +3

Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles

no code implementations CVPR 2024 Rui Song, Chenwei Liang, Hu Cao, Zhiran Yan, Walter Zimmer, Markus Gross, Andreas Festag, Alois Knoll

Additionally, due to the lack of a collaborative perception dataset designed for semantic occupancy prediction, we augment a current collaborative perception dataset to include 3D collaborative semantic occupancy labels for a more robust evaluation.

3D Semantic Occupancy Prediction Prediction

GarchingSim: An Autonomous Driving Simulator with Photorealistic Scenes and Minimalist Workflow

1 code implementation28 Jan 2024 Liguo Zhou, Yinglei Song, Yichao Gao, Zhou Yu, Michael Sodamin, Hongshen Liu, Liang Ma, Lian Liu, Hao liu, Yang Liu, Haichuan Li, Guang Chen, Alois Knoll

However, the availability of free and open-source simulators is limited, and the installation and configuration process can be daunting for beginners and interdisciplinary researchers.

Autonomous Driving

Contact Energy Based Hindsight Experience Prioritization

no code implementations5 Dec 2023 Erdi Sayar, Zhenshan Bing, Carlo D'Eramo, Ozgur S. Oguz, Alois Knoll

Multi-goal robot manipulation tasks with sparse rewards are difficult for reinforcement learning (RL) algorithms due to the inefficiency in collecting successful experiences.

Reinforcement Learning (RL) Robot Manipulation

User-Like Bots for Cognitive Automation: A Survey

no code implementations20 Nov 2023 Habtom Kahsay Gidey, Peter Hillmann, Andreas Karcher, Alois Knoll

In this survey, we aim to explore the role of cognitive architectures in supporting efforts towards engineering software bots with advanced general intelligence.

Survey

Transformation Decoupling Strategy based on Screw Theory for Deterministic Point Cloud Registration with Gravity Prior

1 code implementation2 Nov 2023 Xinyi Li, Zijian Ma, Yinlong Liu, Walter Zimmer, Hu Cao, Feihu Zhang, Alois Knoll

This paper focuses on addressing the robust correspondence-based registration problem with gravity prior that often arises in practice.

Point Cloud Registration

YOLO-BEV: Generating Bird's-Eye View in the Same Way as 2D Object Detection

no code implementations26 Oct 2023 Chang Liu, Liguo Zhou, Yanliang Huang, Alois Knoll

Vehicle perception systems strive to achieve comprehensive and rapid visual interpretation of their surroundings for improved safety and navigation.

Autonomous Driving object-detection +1

Integrate-and-fire circuit for converting analog signals to spikes using phase encoding

no code implementations3 Oct 2023 Javier Lopez-Randulfe, Nico Reeb, Alois Knoll

Thus, we provide an end-to-end neuromorphic application that generates the frequency spectrum of an electric signal without the need for an ADC or a digital signal processing algorithm.

State Representations as Incentives for Reinforcement Learning Agents: A Sim2Real Analysis on Robotic Grasping

1 code implementation21 Sep 2023 Panagiotis Petropoulakis, Ludwig Gräf, Mohammadhossein Malmir, Josip Josifovski, Alois Knoll

The effects of each representation on the ability of the agent to solve the task in simulation and the transferability of the learned policy to the real robot are examined and compared against a model-based approach with complete system knowledge.

Decision Making reinforcement-learning +3

BEA: Revisiting anchor-based object detection DNN using Budding Ensemble Architecture

no code implementations14 Sep 2023 Syed Sha Qutub, Neslihan Kose, Rafael Rosales, Michael Paulitsch, Korbinian Hagn, Florian Geissler, Yang Peng, Gereon Hinz, Alois Knoll

The proposed loss functions in BEA improve the confidence score calibration and lower the uncertainty error, which results in a better distinction of true and false positives and, eventually, higher accuracy of the object detection models.

object-detection Object Detection +1

Strip Attention for Image Restoration

1 code implementation IJCAI 2023 Yuning Cui, Yi Tao, Luoxi Jing, Alois Knoll

As a long-standing task, image restoration aims to recover the latent sharp image from its degraded counterpart.

Image Dehazing Image Restoration +1

DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control

no code implementations15 Jun 2023 Mohammadhossein Malmir, Josip Josifovski, Noah Klarmann, Alois Knoll

We introduce a disturbance-augmented Markov decision process in delayed settings as a novel representation to incorporate disturbance estimation in training on-policy reinforcement learning algorithms.

reinforcement-learning Reinforcement Learning

Language-Conditioned Imitation Learning with Base Skill Priors under Unstructured Data

no code implementations30 May 2023 Hongkuan Zhou, Zhenshan Bing, Xiangtong Yao, Xiaojie Su, Chenguang Yang, Kai Huang, Alois Knoll

In this evaluation, we set up ten tasks and achieved an average 30% improvement in our approach compared to the current state-of-the-art approach, demonstrating a high generalization capability in both simulated environments and the real world.

Imitation Learning Robot Manipulation

Towards Cognitive Bots: Architectural Research Challenges

no code implementations26 May 2023 Habtom Kahsay Gidey, Peter Hillmann, Andreas Karcher, Alois Knoll

Software bots operating in multiple virtual digital platforms must understand the platforms' affordances and behave like human users.

DIVA: A Dirichlet Process Mixtures Based Incremental Deep Clustering Algorithm via Variational Auto-Encoder

1 code implementation23 May 2023 Zhenshan Bing, Yuan Meng, Yuqi Yun, Hang Su, Xiaojie Su, Kai Huang, Alois Knoll

Generative model-based deep clustering frameworks excel in classifying complex data, but are limited in handling dynamic and complex features because they require prior knowledge of the number of clusters.

Clustering Image Generation +2

Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning

no code implementations29 Apr 2023 Mingyang Wang, Zhenshan Bing, Xiangtong Yao, Shuai Wang, Hang Su, Chenguang Yang, Kai Huang, Alois Knoll

On MuJoCo and Meta-World benchmarks, MoSS outperforms prior works in terms of asymptotic performance, sample efficiency (3-50x faster), adaptation efficiency, and generalization robustness on broad and diverse task distributions.

Meta Reinforcement Learning MuJoCo +3

FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems

1 code implementation4 Apr 2023 Rui Song, Runsheng Xu, Andreas Festag, Jiaqi Ma, Alois Knoll

Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving.

Autonomous Driving Federated Learning

TMA: Temporal Motion Aggregation for Event-based Optical Flow

1 code implementation ICCV 2023 Haotian Liu, Guang Chen, Sanqing Qu, Yanping Zhang, Zhijun Li, Alois Knoll, Changjun Jiang

In this paper, we argue that temporal continuity is a vital element of event-based optical flow and propose a novel Temporal Motion Aggregation (TMA) approach to unlock its potential.

Event-based Optical Flow Optical Flow Estimation

Sequential Spatial Network for Collision Avoidance in Autonomous Driving

no code implementations12 Mar 2023 Haichuan Li, Liguo Zhou, Zhenshan Bing, Marzana Khatun, Rolf Jung, Alois Knoll

Several autonomous driving strategies have been applied to autonomous vehicles, especially in the collision avoidance area.

Autonomous Driving Collision Avoidance +1

BCSSN: Bi-direction Compact Spatial Separable Network for Collision Avoidance in Autonomous Driving

no code implementations12 Mar 2023 Haichuan Li, Liguo Zhou, Alois Knoll

In this paper, we propose a CNN-based method that overcomes the limitation by establishing feature correlations between regions in sequential images using variants of attention.

Autonomous Driving Collision Avoidance +2

A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors

no code implementations25 Feb 2023 Shangding Gu, Alap Kshirsagar, Yali Du, Guang Chen, Jan Peters, Alois Knoll

Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment.

reinforcement-learning Reinforcement Learning (RL) +1

Focal Network for Image Restoration

1 code implementation ICCV 2023 Yuning Cui, Wenqi Ren, Xiaochun Cao, Alois Knoll

Image restoration aims to reconstruct a sharp image from its degraded counterpart, which plays an important role in many fields.

Deblurring Image Defocus Deblurring +2

Autonomous Driving Simulator based on Neurorobotics Platform

no code implementations31 Dec 2022 Wei Cao, Liguo Zhou, Yuhong Huang, Alois Knoll

There are many artificial intelligence algorithms for autonomous driving, but directly installing these algorithms on vehicles is unrealistic and expensive.

Autonomous Driving object-detection +1

ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals

no code implementations11 Dec 2022 Rui Song, Liguo Zhou, Lingjuan Lyu, Andreas Festag, Alois Knoll

To address this bottleneck, we introduce a residual-based federated learning framework (ResFed), where residuals rather than model parameters are transmitted in communication networks for training.

Federated Learning Quantization

Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs

1 code implementation7 Sep 2022 Syed Qutub, Florian Geissler, Yang Peng, Ralf Grafe, Michael Paulitsch, Gereon Hinz, Alois Knoll

The evaluation of several representative object detection models shows that even a single bit flip can lead to a severe silent data corruption event with potentially critical safety implications, with e. g., up to (much greater than) 100 FPs generated, or up to approx.

Object object-detection +1

Accurate and Real-time Pseudo Lidar Detection: Is Stereo Neural Network Really Necessary?

no code implementations28 Jun 2022 Haitao Meng, Changcai Li, Gang Chen, Alois Knoll

In the experiments, we develop a system with a less powerful stereo matching predictor and adopt the proposed refinement schemes to improve the accuracy.

3D Object Detection Object +3

Edge-Aided Sensor Data Sharing in Vehicular Communication Networks

no code implementations17 Jun 2022 Rui Song, Anupama Hegde, Numan Senel, Alois Knoll, Andreas Festag

Specifically, when the measurement error from the sensors (also referred as measurement noise) is unknown and time varying, the performance of the data fusion process is restricted, which represents a major challenge in the calibration of sensors.

Noise Estimation

A Review of Safe Reinforcement Learning: Methods, Theory and Applications

1 code implementation20 May 2022 Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, Alois Knoll

To establish a good foundation for future safe RL research, in this paper, we provide a review of safe RL from the perspectives of methods, theories, and applications.

Autonomous Driving Decision Making +4

3D Object Detection with a Self-supervised Lidar Scene Flow Backbone

2 code implementations2 May 2022 Ekim Yurtsever, Emeç Erçelik, MingYu Liu, Zhijie Yang, Hanzhen Zhang, Pınar Topçam, Maximilian Listl, Yılmaz Kaan Çaylı, Alois Knoll

Our main contribution leverages learned flow and motion representations and combines a self-supervised backbone with a supervised 3D detection head.

3D Object Detection Object +3

A9-Dataset: Multi-Sensor Infrastructure-Based Dataset for Mobility Research

no code implementations13 Apr 2022 Christian Creß, Walter Zimmer, Leah Strand, Venkatnarayanan Lakshminarasimhan, Maximilian Fortkord, Siyi Dai, Alois Knoll

As part of the first set of data, which we describe in this paper, we provide camera and LiDAR frames from two overhead gantry bridges on the A9 autobahn with the corresponding objects labeled with 3D bounding boxes.

Management

Federated Learning Framework Coping with Hierarchical Heterogeneity in Cooperative ITS

1 code implementation1 Apr 2022 Rui Song, Liguo Zhou, Venkatnarayanan Lakshminarasimhan, Andreas Festag, Alois Knoll

Considering the individual heterogeneity of data distribution, computational and communication capabilities across traffic agents and roadside units, we employ a novel method that addresses the heterogeneity of different aggregation layers of the framework architecture, i. e., aggregation in layers of roadside units and cloud.

Autonomous Vehicles Federated Learning

A Survey of Robust 3D Object Detection Methods in Point Clouds

no code implementations31 Mar 2022 Walter Zimmer, Emec Ercelik, Xingcheng Zhou, Xavier Jair Diaz Ortiz, Alois Knoll

The purpose of this work is to review the state-of-the-art LiDAR-based 3D object detection methods, datasets, and challenges.

Autonomous Driving Data Augmentation +3

Real-Time and Robust 3D Object Detection Within Road-Side LiDARs Using Domain Adaptation

no code implementations31 Mar 2022 Walter Zimmer, Marcus Grabler, Alois Knoll

This work aims to address the challenges in domain adaptation of 3D object detection using infrastructure LiDARs.

Domain Adaptation object-detection +2

Graph Neural Networks for Relational Inductive Bias in Vision-based Deep Reinforcement Learning of Robot Control

no code implementations11 Mar 2022 Marco Oliva, Soubarna Banik, Josip Josifovski, Alois Knoll

We derive a graph representation that models the physical structure of the manipulator and combines the robot's internal state with a low-dimensional description of the visual scene generated by an image encoding network.

Deep Reinforcement Learning Graph Neural Network +3

Are Transformers More Robust? Towards Exact Robustness Verification for Transformers

no code implementations8 Feb 2022 Brian Hsuan-Cheng Liao, Chih-Hong Cheng, Hasan Esen, Alois Knoll

As an emerging type of Neural Networks (NNs), Transformers are used in many domains ranging from Natural Language Processing to Autonomous Driving.

Autonomous Driving Object Recognition

Evaluating Muscle Synergies with EMG Data and Physics Simulation in the Neurorobotics Platform

no code implementations14 Jan 2022 Benedikt Feldotto, Cristian Soare, Alois Knoll, Piyanee Sriya, Sarah Astill, Marc de Kamps, Samit Chakrabarty

We use our framework to analyze raw EMG data collected during an isometric knee extension study to identify synergies that drive a musculoskeletal lower limb model.

Electromyography (EMG)

Multi-Agent Constrained Policy Optimisation

4 code implementations6 Oct 2021 Shangding Gu, Jakub Grudzien Kuba, Munning Wen, Ruiqing Chen, Ziyan Wang, Zheng Tian, Jun Wang, Alois Knoll, Yaodong Yang

To fill these gaps, in this work, we formulate the safe MARL problem as a constrained Markov game and solve it with policy optimisation methods.

MuJoCo Multi-agent Reinforcement Learning +3

Towards Cognitive Navigation: Design and Implementation of a Biologically Inspired Head Direction Cell Network

no code implementations22 Sep 2021 Zhenshan Bing, Amir EI Sewisy, Genghang Zhuang, Florian Walter, Fabrice O. Morin, Kai Huang, Alois Knoll

As a vital cognitive function of animals, the navigation skill is first built on the accurate perception of the directional heading in the environment.

Vogtareuth Rehab Depth Datasets: Benchmark for Marker-less Posture Estimation in Rehabilitation

no code implementations23 Aug 2021 Soubarna Banik, Alejandro Mendoza Garcia, Lorenz Kiwull, Steffen Berweck, Alois Knoll

We evaluate it on our rehab datasets, and observe that the performance degrades significantly from non-rehab to rehab, highlighting the need for these datasets.

Meta-Reinforcement Learning in Broad and Non-Parametric Environments

1 code implementation8 Aug 2021 Zhenshan Bing, Lukas Knak, Fabrice Oliver Robin, Kai Huang, Alois Knoll

Recent state-of-the-art artificial agents lack the ability to adapt rapidly to new tasks, as they are trained exclusively for specific objectives and require massive amounts of interaction to learn new skills.

Meta Reinforcement Learning reinforcement-learning +2

Safety Metrics for Semantic Segmentation in Autonomous Driving

no code implementations21 May 2021 Chih-Hong Cheng, Alois Knoll, Hsuan-Cheng Liao

Within the context of autonomous driving, safety-related metrics for deep neural networks have been widely studied for image classification and object detection.

Autonomous Driving Clustering +5

Temp-Frustum Net: 3D Object Detection with Temporal Fusion

1 code implementation25 Apr 2021 Emeç Erçelik, Ekim Yurtsever, Alois Knoll

Furthermore, ablation studies reinforce that the subject of improvement is temporal fusion and show the effects of different placements of TFM in the object detection pipeline.

3D Object Detection Object +2

A Hierarchical State-Machine-Based Framework for Platoon Manoeuvre Descriptions

no code implementations12 Apr 2021 Corvin Deboeser, Jordan Ivanchev, Thomas Braud, Alois Knoll, David Eckhoff, Alberto Sangiovanni-Vincentelli

This paper introduces the SEAD framework that simplifies the process of designing and describing autonomous vehicle platooning manoeuvres.

ACM-Net: Action Context Modeling Network for Weakly-Supervised Temporal Action Localization

2 code implementations7 Apr 2021 Sanqing Qu, Guang Chen, Zhijun Li, Lijun Zhang, Fan Lu, Alois Knoll

Traditional methods mainly focus on foreground and background frames separation with only a single attention branch and class activation sequence.

Weakly Supervised Action Localization

Non-Holonomic RRT & MPC: Path and Trajectory Planning for an Autonomous Cycle Rickshaw

no code implementations10 Mar 2021 Damir Bojadžić, Julian Kunze, Dinko Osmanković, Mohammadhossein Malmir, Alois Knoll

Therefore, the algorithm presented in this paper needs to anticipate and avoid dynamic obstacles, such as pedestrians or bicycles, but also be fast enough in order to work in real-time so that it can adapt to changes in the environment.

Motion Planning Trajectory Planning Robotics

FloMo: Tractable Motion Prediction with Normalizing Flows

1 code implementation5 Mar 2021 Christoph Schöller, Alois Knoll

In our work, we model motion prediction directly as a density estimation problem with a normalizing flow between a noise distribution and the future motion distribution.

Data Augmentation Density Estimation +2

NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting

1 code implementation10 Feb 2021 Kai Chen, Guang Chen, Dan Xu, Lijun Zhang, Yuyao Huang, Alois Knoll

Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge.

Time Series Time Series Forecasting

Experience-Based Heuristic Search: Robust Motion Planning with Deep Q-Learning

no code implementations5 Feb 2021 Julian Bernhard, Robert Gieselmann, Klemens Esterle, Alois Knoll

With Deep Reinforcement Learning, optimal driving strategies for such problems can be derived also for higher-dimensional problems.

Autonomous Driving Deep Reinforcement Learning +5

Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection

no code implementations25 Jan 2021 Hu Cao, Guang Chen, Zhijun Li, Jianjie Lin, Alois Knoll

Extensive experiments on two public grasping datasets, Cornell and Jacquard demonstrate the state-of-the-art performance of our method in balancing accuracy and inference speed.

object-detection Robotic Grasping

PointINet: Point Cloud Frame Interpolation Network

1 code implementation18 Dec 2020 Fan Lu, Guang Chen, Sanqing Qu, Zhijun Li, Yinlong Liu, Alois Knoll

Generally, the frame rates of mechanical LiDAR sensors are 10 to 20 Hz, which is much lower than other commonly used sensors like cameras.

3D Point Cloud Interpolation

LAP-Net: Adaptive Features Sampling via Learning Action Progression for Online Action Detection

no code implementations16 Nov 2020 Sanqing Qu, Guang Chen, Dan Xu, Jinhu Dong, Fan Lu, Alois Knoll

At each time step, this sampling strategy first estimates current action progression and then decide what temporal ranges should be used to aggregate the optimal supplementary features.

Online Action Detection

RSKDD-Net: Random Sample-based Keypoint Detector and Descriptor

1 code implementation NeurIPS 2020 Fan Lu, Guang Chen, Yinlong Liu, Zhongnan Qu, Alois Knoll

To tackle the information loss of random sampling, we exploit a novel random dilation cluster strategy to enlarge the receptive field of each sampled point and an attention mechanism to aggregate the positions and features of neighbor points.

Point Cloud Registration Saliency Prediction

Neuron Activation Analysis for Multi-Joint Robot Reinforcement Learning

no code implementations28 Sep 2020 Benedikt Feldotto, Heiko Lengenfelder, Alois Knoll

We analyze the individual neuron activity distribution in the network, introduce a pruning algorithm to reduce network size keeping the performance, and with these dense network representations we spot correlations of neuron activity patterns among networks trained for robot manipulators with different joint number.

reinforcement-learning Reinforcement Learning +1

Complex Robotic Manipulation via Graph-Based Hindsight Goal Generation

1 code implementation27 Jul 2020 Zhenshan Bing, Matthias Brucker, Fabrice O. Morin, Kai Huang, Alois Knoll

In this paper, we propose graph-based hindsight goal generation (G-HGG), an extension of HGG selecting hindsight goals based on shortest distances in an obstacle-avoiding graph, which is a discrete representation of the environment.

Formalizing Traffic Rules for Machine Interpretability

2 code implementations1 Jul 2020 Klemens Esterle, Luis Gressenbuch, Alois Knoll

We contribute a formalized set of traffic rules for single-direction carriageways, such as on highways.

Robotics

AerialMPTNet: Multi-Pedestrian Tracking in Aerial Imagery Using Temporal and Graphical Features

no code implementations27 Jun 2020 Maximilian Kraus, Seyed Majid Azimi, Emec Ercelik, Reza Bahmanyar, Peter Reinartz, Alois Knoll

Due to the challenges such as the large number and the tiny size of the pedestrians (e. g., 4 x 4 pixels) with their similar appearances as well as different scales and atmospheric conditions of the images with their extremely low frame rates (e. g., 2 fps), current state-of-the-art algorithms including the deep learning-based ones are unable to perform well.

Management

Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

2 code implementations22 Jun 2020 Patrick Hart, Alois Knoll

We show that graph neural networks are capable of handling scenarios with a varying number and order of vehicles during training and application.

reinforcement-learning Reinforcement Learning +1

Counterfactual Policy Evaluation for Decision-Making in Autonomous Driving

2 code implementations20 Mar 2020 Patrick Hart, Alois Knoll

If a policy can handle all counterfactual worlds well, it either has seen similar situations during training or it generalizes well and is deemed to be fit enough to be executed in the actual world.

Autonomous Driving counterfactual +2

Task-Independent Spiking Central Pattern Generator: A Learning-Based Approach

no code implementations17 Mar 2020 Elie Aljalbout, Florian Walter, Florian Röhrbein, Alois Knoll

This model is the main focus of this work, as its contribution is not limited to engineering but also applicable to neuroscience.

Indirect and Direct Training of Spiking Neural Networks for End-to-End Control of a Lane-Keeping Vehicle

no code implementations10 Mar 2020 Zhenshan Bing, Claus Meschede, Guang Chen, Alois Knoll, Kai Huang

Building spiking neural networks (SNNs) based on biological synaptic plasticities holds a promising potential for accomplishing fast and energy-efficient computing, which is beneficial to mobile robotic applications.

Q-Learning Reinforcement Learning

Identity Recognition in Intelligent Cars with Behavioral Data and LSTM-ResNet Classifier

no code implementations2 Mar 2020 Michael Hammann, Maximilian Kraus, Sina Shafaei, Alois Knoll

Identity recognition in a car cabin is a critical task nowadays and offers a great field of applications ranging from personalizing intelligent cars to suit drivers physical and behavioral needs to increasing safety and security.

General Classification Time Series +2

Providentia -- A Large-Scale Sensor System for the Assistance of Autonomous Vehicles and Its Evaluation

no code implementations16 Jun 2019 Annkathrin Krämmer, Christoph Schöller, Dhiraj Gulati, Venkatnarayanan Lakshminarasimhan, Franz Kurz, Dominik Rosenbaum, Claus Lenz, Alois Knoll

An Intelligent Infrastructure System can fill in the gaps in a vehicle's perception and extend its field of view by providing additional detailed information about its surroundings, in the form of a digital model of the current traffic situation, i. e. a digital twin.

Autonomous Vehicles Earth Observation

Copy and Paste: A Simple But Effective Initialization Method for Black-Box Adversarial Attacks

1 code implementation14 Jun 2019 Thomas Brunner, Frederik Diehl, Alois Knoll

Many optimization methods for generating black-box adversarial examples have been proposed, but the aspect of initializing said optimizers has not been considered in much detail.

valid

Leveraging Semantic Embeddings for Safety-Critical Applications

no code implementations19 May 2019 Thomas Brunner, Frederik Diehl, Michael Truong Le, Alois Knoll

Semantic Embeddings are a popular way to represent knowledge in the field of zero-shot learning.

Zero-Shot Learning

Globally optimal vertical direction estimation in Atlanta World

1 code implementation29 Apr 2019 Yinlong Liu, Alois Knoll, Guang Chen

Accordingly, we propose a vertical direction estimation method by considering the relationship between the vertical frame and horizontal frames.

A Novel Method for the Absolute Pose Problem with Pairwise Constraints

no code implementations25 Mar 2019 Yinlong Liu, Xuechen Li, Manning Wang, Guang Chen, Zhijian Song, Alois Knoll

In this paper, we consider pairwise constraints and propose a globally optimal algorithm for solving the absolute pose estimation problem.

parameter estimation Pose Estimation +1

What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction

2 code implementations19 Mar 2019 Christoph Schöller, Vincent Aravantinos, Florian Lay, Alois Knoll

Our work shows how neural networks for pedestrian motion prediction can be thoroughly evaluated and our results indicate which research directions for neural motion prediction are promising in future.

motion prediction Prediction

Graph Neural Networks for Modelling Traffic Participant Interaction

no code implementations4 Mar 2019 Frederik Diehl, Thomas Brunner, Michael Truong Le, Alois Knoll

We show that prediction error in scenarios with much interaction decreases by 30% compared to a model that does not take interactions into account.

Graph Neural Network Prediction +1

Deep Anticipation: Light Weight Intelligent Mobile Sensing in IoT by Recurrent Architecture

no code implementations6 Dec 2017 Guang Chen, Shu Liu, Kejia Ren, Zhongnan Qu, Changhong Fu, Gereon Hinz, Alois Knoll

However, the mobile sensing perception brings new challenges for how to efficiently analyze and intelligently interpret the deluge of IoT data in mission- critical services.

Variational PatchMatch MultiView Reconstruction and Refinement

no code implementations ICCV 2015 Philipp Heise, Brian Jensen, Sebastian Klose, Alois Knoll

We formulate the combined multi-view stereo reconstruction and refinement as a variational optimization problem.

Cannot find the paper you are looking for? You can Submit a new open access paper.