Search Results for author: Stefan Wermter

Found 128 papers, 44 papers with code

Causal State Distillation for Explainable Reinforcement Learning

no code implementations30 Dec 2023 Wenhao Lu, Xufeng Zhao, Thilo Fryen, Jae Hee Lee, Mengdi Li, Sven Magg, Stefan Wermter

This lack of transparency in RL models has been a long-standing problem, making it difficult for users to grasp the reasons behind an agent's behaviour.

reinforcement-learning Reinforcement Learning (RL)

Read Between the Layers: Leveraging Intra-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models

no code implementations13 Dec 2023 Kyra Ahrens, Hans Hergen Lehmann, Jae Hee Lee, Stefan Wermter

We address the Continual Learning (CL) problem, where a model has to learn a sequence of tasks from non-stationary distributions while preserving prior knowledge as it encounters new experiences.

Class Incremental Learning Incremental Learning

Visually Grounded Continual Language Learning with Selective Specialization

1 code implementation24 Oct 2023 Kyra Ahrens, Lennart Bengtson, Jae Hee Lee, Stefan Wermter

Selective specialization, i. e., a careful selection of model components to specialize in each task, is a strategy to provide control over this trade-off.

Continual Learning

From Neural Activations to Concepts: A Survey on Explaining Concepts in Neural Networks

no code implementations18 Oct 2023 Jae Hee Lee, Sergio Lanza, Stefan Wermter

In this paper, we review recent approaches for explaining concepts in neural networks.

Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic

1 code implementation23 Sep 2023 Xufeng Zhao, Mengdi Li, Wenhao Lu, Cornelius Weber, Jae Hee Lee, Kun Chu, Stefan Wermter

Recent advancements in large language models have showcased their remarkable generalizability across various domains.

Causal Inference

Continual Robot Learning using Self-Supervised Task Inference

no code implementations10 Sep 2023 Muhammad Burhan Hafez, Stefan Wermter

Our approach learns action and intention embeddings from self-organization of the observed movement and effect parts of unlabeled demonstrations and a higher-level behavior embedding from self-organization of the joint action-intention embeddings.

Continual Learning Multi-Task Learning +1

Bring the Noise: Introducing Noise Robustness to Pretrained Automatic Speech Recognition

no code implementations5 Sep 2023 Patrick Eickhoff, Matthias Möller, Theresa Pekarek Rosin, Johannes Twiefel, Stefan Wermter

We show that the Cleancoder is able to filter noise from speech and that it improves the total Word Error Rate (WER) of the downstream model in noisy conditions for both applications.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

CycleIK: Neuro-inspired Inverse Kinematics

1 code implementation21 Jul 2023 Jan-Gerrit Habekost, Erik Strahl, Philipp Allgeuer, Matthias Kerzel, Stefan Wermter

The paper introduces CycleIK, a neuro-robotic approach that wraps two novel neuro-inspired methods for the inverse kinematics (IK) task, a Generative Adversarial Network (GAN), and a Multi-Layer Perceptron architecture.

Generative Adversarial Network

Clarifying the Half Full or Half Empty Question: Multimodal Container Classification

no code implementations17 Jul 2023 Josua Spisak, Matthias Kerzel, Stefan Wermter

Multimodal integration is a key component of allowing robots to perceive the world.

Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognition

no code implementations14 Jul 2023 Theresa Pekarek Rosin, Stefan Wermter

While Automatic Speech Recognition (ASR) models have shown significant advances with the introduction of unsupervised or self-supervised training techniques, these improvements are still only limited to a subsection of languages and speakers.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Integrating Uncertainty into Neural Network-based Speech Enhancement

1 code implementation15 May 2023 Huajian Fang, Dennis Becker, Stefan Wermter, Timo Gerkmann

In this paper, we study the benefits of modeling uncertainty in clean speech estimation.

Speech Enhancement

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

1 code implementation3 May 2023 Muhammad Burhan Hafez, Tilman Immisch, Tom Weber, Stefan Wermter

Our approach organizes stored transitions into a concise environment-model-like network of state-nodes and transition-edges, merging similar samples to reduce the memory size and increase pair-wise distance among samples, which increases the relevancy of each sample.

A Parameter-free Adaptive Resonance Theory-based Topological Clustering Algorithm Capable of Continual Learning

1 code implementation1 May 2023 Naoki Masuyama, Takanori Takebayashi, Yusuke Nojima, Chu Kiong Loo, Hisao Ishibuchi, Stefan Wermter

In general, a similarity threshold (i. e., a vigilance parameter) for a node learning process in Adaptive Resonance Theory (ART)-based algorithms has a significant impact on clustering performance.

Clustering Continual Learning

A Closer Look at Reward Decomposition for High-Level Robotic Explanations

no code implementations25 Apr 2023 Wenhao Lu, Xufeng Zhao, Sven Magg, Martin Gromniak, Mengdi Li, Stefan Wermter

Explaining the behaviour of intelligent agents learned by reinforcement learning (RL) to humans is challenging yet crucial due to their incomprehensible proprioceptive states, variational intermediate goals, and resultant unpredictability.

Reinforcement Learning (RL) Vocal Bursts Intensity Prediction

Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise

no code implementations27 Mar 2023 Huajian Fang, Niklas Wittmer, Johannes Twiefel, Stefan Wermter, Timo Gerkmann

In this paper, we propose a multichannel partially adaptive scheme to jointly model ego-noise and environmental noise utilizing the VAE-NMF framework, where we take advantage of spatially and spectrally structured characteristics of ego-noise by pre-training the ego-noise model, while retaining the ability to adapt to unknown environmental noise.

Chat with the Environment: Interactive Multimodal Perception Using Large Language Models

1 code implementation14 Mar 2023 Xufeng Zhao, Mengdi Li, Cornelius Weber, Muhammad Burhan Hafez, Stefan Wermter

However, it remains challenging to ground LLMs in multimodal sensory input and continuous action output, while enabling a robot to interact with its environment and acquire novel information as its policies unfold.

Sample-efficient Real-time Planning with Curiosity Cross-Entropy Method and Contrastive Learning

1 code implementation7 Mar 2023 Mostafa Kotb, Cornelius Weber, Stefan Wermter

Model-based reinforcement learning (MBRL) with real-time planning has shown great potential in locomotion and manipulation control tasks.

Continuous Control Contrastive Learning +2

Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition

no code implementations20 Feb 2023 Leyuan Qu, Cornelius Weber, Stefan Wermter

Furthermore, our proposed combined loss rescaling and weight consolidation methods can support continual learning of an ASR system.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +5

Internally Rewarded Reinforcement Learning

1 code implementation1 Feb 2023 Mengdi Li, Xufeng Zhao, Jae Hee Lee, Cornelius Weber, Stefan Wermter

We study a class of reinforcement learning problems where the reward signals for policy learning are generated by an internal reward model that is dependent on and jointly optimized with the policy.

reinforcement-learning Reinforcement Learning (RL)

Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations

1 code implementation8 Dec 2022 Björn Plüster, Jakob Ambsdorf, Lukas Braach, Jae Hee Lee, Stefan Wermter

Natural language explanations promise to offer intuitively understandable explanations of a neural network's decision process in complex vision-language tasks, as pursued in recent VL-NLE models.

Explanation Generation Visual Entailment +1

Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge

1 code implementation23 Nov 2022 Hugo Carneiro, Cornelius Weber, Stefan Wermter

Finally, we devise a model for emotion recognition in conversations trained on the realigned MELD-FAIR videos, which outperforms state-of-the-art models for ERC based on vision alone.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Introspection-based Explainable Reinforcement Learning in Episodic and Non-episodic Scenarios

no code implementations23 Nov 2022 Niclas Schroeter, Francisco Cruz, Stefan Wermter

Results obtained show the viability of introspection for episodic robotics tasks and, additionally, that the introspection-based approach can be used to generate explanations for the actions taken in a non-episodic robotics environment as well.

reinforcement-learning Reinforcement Learning (RL)

Visually Grounded Commonsense Knowledge Acquisition

1 code implementation22 Nov 2022 Yuan YAO, Tianyu Yu, Ao Zhang, Mengdi Li, Ruobing Xie, Cornelius Weber, Zhiyuan Liu, Hai-Tao Zheng, Stefan Wermter, Tat-Seng Chua, Maosong Sun

In this work, we present CLEVER, which formulates CKE as a distantly supervised multi-instance learning problem, where models learn to summarize commonsense relations from a bag of images about an entity pair without any human annotation on image instances.

Language Modelling

Improving Speech Emotion Recognition with Unsupervised Speaking Style Transfer

no code implementations16 Nov 2022 Leyuan Qu, Wei Wang, Cornelius Weber, Pengcheng Yue, Taihao Li, Stefan Wermter

Once training is completed, EmoAug enriches expressions of emotional speech with different prosodic attributes, such as stress, rhythm and intensity, by feeding different styles into the paralinguistic encoder.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Learning to Autonomously Reach Objects with NICO and Grow-When-Required Networks

no code implementations14 Oct 2022 Nima Rahrakhshan, Matthias Kerzel, Philipp Allgeuer, Nicolas Duczek, Stefan Wermter

The act of reaching for an object is a fundamental yet complex skill for a robotic agent, requiring a high degree of visuomotor control and coordination.

Object

Intelligent problem-solving as integrated hierarchical reinforcement learning

no code implementations18 Aug 2022 Manfred Eppe, Christian Gumbsch, Matthias Kerzel, Phuong D. H. Nguyen, Martin V. Butz, Stefan Wermter

According to cognitive psychology and related disciplines, the development of complex problem-solving behaviour in biological agents depends on hierarchical cognitive mechanisms.

Hierarchical Reinforcement Learning reinforcement-learning +1

Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations

1 code implementation4 Aug 2022 Xufeng Zhao, Cornelius Weber, Muhammad Burhan Hafez, Stefan Wermter

Sound is one of the most informative and abundant modalities in the real world while being robust to sense without contacts by small and cheap sensors that can be placed on mobile devices.

Efficient Exploration Unsupervised Reinforcement Learning

Learning Flexible Translation between Robot Actions and Language Descriptions

no code implementations15 Jul 2022 Ozan Özdemir, Matthias Kerzel, Cornelius Weber, Jae Hee Lee, Stefan Wermter

In this work, we propose the paired gated autoencoders (PGAE) for flexible translation between robot actions and language descriptions in a tabletop object manipulation scenario.

Language Modelling Multi-Task Learning +1

Knowing Earlier what Right Means to You: A Comprehensive VQA Dataset for Grounding Relative Directions via Multi-Task Learning

1 code implementation6 Jul 2022 Kyra Ahrens, Matthias Kerzel, Jae Hee Lee, Cornelius Weber, Stefan Wermter

Spatial reasoning poses a particular challenge for intelligent agents and is at the same time a prerequisite for their successful interaction and communication in the physical world.

Multi-Task Learning Question Answering +1

GASP: Gated Attention For Saliency Prediction

1 code implementation International Joint Conference on Artificial Intelligence 2021 Fares Abawi, Tom Weber, Stefan Wermter

We show that gaze direction and affective representations contribute a prediction to ground-truth correspondence improvement of at least 5% compared to dynamic saliency models without social cues.

Saliency Prediction Video Saliency Detection +1

Snapture -- A Novel Neural Architecture for Combined Static and Dynamic Hand Gesture Recognition

1 code implementation28 May 2022 Hassan Ali, Doreen Jirak, Stefan Wermter

Our architecture enables learning both static and dynamic gestures: by capturing a so-called "snapshot" of the gesture performance at its peak, we integrate the hand pose along with the dynamic movement.

Hand Gesture Recognition Hand-Gesture Recognition

Conversational Analysis of Daily Dialog Data using Polite Emotional Dialogue Acts

no code implementations LREC 2022 Chandrakant Bothe, Stefan Wermter

One of the fundamental cues is politeness, which linguistically possesses properties such as social manners useful in conversational analysis.

Emotional Dialogue Acts

What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning

1 code implementation5 May 2022 Jae Hee Lee, Matthias Kerzel, Kyra Ahrens, Cornelius Weber, Stefan Wermter

Grounding relative directions is more difficult than grounding absolute directions because it not only requires a model to detect objects in the image and to identify spatial relation based on this information, but it also needs to recognize the orientation of objects and integrate this information into the reasoning process.

Multi-Task Learning Question Answering +1

Integrating Statistical Uncertainty into Neural Network-Based Speech Enhancement

no code implementations4 Mar 2022 Huajian Fang, Tal Peer, Stefan Wermter, Timo Gerkmann

Speech enhancement in the time-frequency domain is often performed by estimating a multiplicative mask to extract clean speech.

Speech Enhancement

Language Model-Based Paired Variational Autoencoders for Robotic Language Learning

no code implementations17 Jan 2022 Ozan Özdemir, Matthias Kerzel, Cornelius Weber, Jae Hee Lee, Stefan Wermter

Human infants learn language while interacting with their environment in which their caregivers may describe the objects and actions they perform.

Language Modelling

LipSound2: Self-Supervised Pre-Training for Lip-to-Speech Reconstruction and Lip Reading

no code implementations9 Dec 2021 Leyuan Qu, Cornelius Weber, Stefan Wermter

The aim of this work is to investigate the impact of crossmodal self-supervised pre-training for speech reconstruction (video-to-audio) by leveraging the natural co-occurrence of audio and visual streams in videos.

Lip Reading speech-recognition +1

Lifelong Learning from Event-based Data

1 code implementation11 Nov 2021 Vadym Gryshchuk, Cornelius Weber, Chu Kiong Loo, Stefan Wermter

Lifelong learning is a long-standing aim for artificial agents that act in dynamic environments, in which an agent needs to accumulate knowledge incrementally without forgetting previously learned representations.

A trained humanoid robot can perform human-like crossmodal social attention and conflict resolution

no code implementations2 Nov 2021 Di Fu, Fares Abawi, Hugo Carneiro, Matthias Kerzel, Ziwei Chen, Erik Strahl, Xun Liu, Stefan Wermter

Our saliency prediction model was trained to detect social cues, predict audio-visual saliency, and attend selectively for the robot study.

Saliency Prediction

Can AI detect pain and express pain empathy? A review from emotion recognition and a human-centered AI perspective

no code implementations8 Oct 2021 Siqi Cao, Di Fu, Xu Yang, Stefan Wermter, Xun Liu, Haiyan Wu

Furthermore, we discuss challenges for responsible evaluation of cognitive methods and computational techniques and show approaches to future work to contribute to affective assistants capable of empathy.

Emotion Recognition

FaVoA: Face-Voice Association Favours Ambiguous Speaker Detection

no code implementations1 Sep 2021 Hugo Carneiro, Cornelius Weber, Stefan Wermter

The strong relation between face and voice can aid active speaker detection systems when faces are visible, even in difficult settings, when the face of a speaker is not clear or when there are several people in the same scene.

Generalization in Multimodal Language Learning from Simulation

no code implementations3 Aug 2021 Aaron Eisermann, Jae Hee Lee, Cornelius Weber, Stefan Wermter

Neural networks can be powerful function approximators, which are able to model high-dimensional feature distributions from a subset of examples drawn from the target distribution.

Behavior Self-Organization Supports Task Inference for Continual Robot Learning

no code implementations9 Jul 2021 Muhammad Burhan Hafez, Stefan Wermter

Task inference is made by finding the nearest behavior embedding to a demonstrated behavior, which is used together with the environment state as input to a multi-task policy trained with reinforcement learning to optimize performance over tasks.

Continual Learning Multi-Task Learning

DRILL: Dynamic Representations for Imbalanced Lifelong Learning

1 code implementation18 May 2021 Kyra Ahrens, Fares Abawi, Stefan Wermter

Continual or lifelong learning has been a long-standing challenge in machine learning to date, especially in natural language processing (NLP).

Continual Learning Meta-Learning +2

Survey on reinforcement learning for language processing

no code implementations12 Apr 2021 Victor Uc-Cetina, Nicolas Navarro-Guerrero, Anabel Martin-Gonzalez, Cornelius Weber, Stefan Wermter

In recent years some researchers have explored the use of reinforcement learning (RL) algorithms as key components in the solution of various natural language processing tasks.

reinforcement-learning Reinforcement Learning (RL)

Visual Distant Supervision for Scene Graph Generation

1 code implementation ICCV 2021 Yuan YAO, Ao Zhang, Xu Han, Mengdi Li, Cornelius Weber, Zhiyuan Liu, Stefan Wermter, Maosong Sun

In this work, we propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data.

Graph Generation Predicate Classification +2

A Sub-Layered Hierarchical Pyramidal Neural Architecture for Facial Expression Recognition

no code implementations23 Mar 2021 Henrique Siqueira, Pablo Barros, Sven Magg, Cornelius Weber, Stefan Wermter

In domains where computational resources and labeled data are limited, such as in robotics, deep networks with millions of weights might not be the optimal solution.

Facial Expression Recognition Facial Expression Recognition (FER)

An Ensemble with Shared Representations Based on Convolutional Networks for Continually Learning Facial Expressions

no code implementations5 Mar 2021 Henrique Siqueira, Pablo Barros, Sven Magg, Stefan Wermter

Social robots able to continually learn facial expressions could progressively improve their emotion recognition capability towards people interacting with them.

Emotion Recognition

Disambiguating Affective Stimulus Associations for Robot Perception and Dialogue

no code implementations5 Mar 2021 Henrique Siqueira, Alexander Sutherland, Pablo Barros, Mattias Kerzel, Sven Magg, Stefan Wermter

In this paper, we utilize the NICO robot's appearance and capabilities to give the NICO the ability to model a coherent affective association between a perceived auditory stimulus and a temporally asynchronous emotion expression.

Continual Learning from Synthetic Data for a Humanoid Exercise Robot

no code implementations19 Feb 2021 Nicolas Duczek, Matthias Kerzel, Stefan Wermter

In a practical scenario, a physical exercise is performed by an expert like a physiotherapist and then used as a reference for a humanoid robot like Pepper to give feedback on a patient's execution of the same exercise.

Continual Learning

Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder

no code implementations17 Feb 2021 Huajian Fang, Guillaume Carbajal, Stefan Wermter, Timo Gerkmann

Recently, a generative variational autoencoder (VAE) has been proposed for speech enhancement to model speech statistics.

Speech Enhancement

Improving Model-Based Reinforcement Learning with Internal State Representations through Self-Supervision

1 code implementation10 Feb 2021 Julien Scholz, Cornelius Weber, Muhammad Burhan Hafez, Stefan Wermter

Using a model of the environment, reinforcement learning agents can plan their future moves and achieve superhuman performance in board games like Chess, Shogi, and Go, while remaining relatively sample-efficient.

Board Games Model-based Reinforcement Learning +3

CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

1 code implementation5 Feb 2021 Tobias Hinz, Matthew Fisher, Oliver Wang, Eli Shechtman, Stefan Wermter

Our model generates novel poses based on keypoint locations, which can be modified in real time while providing interactive feedback, allowing for intuitive reposing and animation.

Hierarchical principles of embodied reinforcement learning: A review

no code implementations18 Dec 2020 Manfred Eppe, Christian Gumbsch, Matthias Kerzel, Phuong D. H. Nguyen, Martin V. Butz, Stefan Wermter

We then relate these insights with contemporary hierarchical reinforcement learning methods, and identify the key machine intelligence approaches that realise these mechanisms.

Hierarchical Reinforcement Learning reinforcement-learning +1

Sensorimotor representation learning for an "active self" in robots: A model survey

no code implementations25 Nov 2020 Phuong D. H. Nguyen, Yasmin Kim Georgie, Ezgi Kayhan, Manfred Eppe, Verena Vanessa Hafner, Stefan Wermter

Safe human-robot interactions require robots to be able to learn how to behave appropriately in \sout{humans' world} \rev{spaces populated by people} and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations.

Representation Learning

Affect-Driven Modelling of Robot Personality for Collaborative Human-Robot Interactions

no code implementations14 Oct 2020 Nikhil Churamani, Pablo Barros, Hatice Gunes, Stefan Wermter

Collaborative interactions require social robots to adapt to the dynamics of human affective behaviour.

Enhancing a Neurocognitive Shared Visuomotor Model for Object Identification, Localization, and Grasping With Learning From Auxiliary Tasks

1 code implementation26 Sep 2020 Matthias Kerzel, Fares Abawi, Manfred Eppe, Stefan Wermter

In this follow-up study, we expand the task and the model to reaching for objects in a three-dimensional space with a novel dataset based on augmented reality and a simulation environment.

Crossmodal Language Grounding in an Embodied Neurocognitive Model

1 code implementation24 Jun 2020 Stefan Heinrich, Yuan YAO, Tobias Hinz, Zhiyuan Liu, Thomas Hummel, Matthias Kerzel, Cornelius Weber, Stefan Wermter

From a neuroscientific perspective, natural language is embodied, grounded in most, if not all, sensory and sensorimotor modalities, and acquired by means of crossmodal integration.

Facial Expression Editing with Continuous Emotion Labels

no code implementations22 Jun 2020 Alexandra Lindt, Pablo Barros, Henrique Siqueira, Stefan Wermter

Recently deep generative models have achieved impressive results in the field of automated facial expression editing.

Towards a self-organizing pre-symbolic neural model representing sensorimotor primitives

no code implementations20 Jun 2020 Junpei Zhong, Angelo Cangelosi, Stefan Wermter

During the learning process of observing sensorimotor primitives, i. e. observing a set of trajectories of arm movements and its oriented object features, the pre-symbolic representation is self-organized in the parametric units.

Multimodal Target Speech Separation with Voice and Face References

no code implementations17 May 2020 Leyuan Qu, Cornelius Weber, Stefan Wermter

Target speech separation refers to isolating target speech from a multi-speaker mixture signal by conditioning on auxiliary information about the target speaker.

Audio and Speech Processing Sound

Curious Hierarchical Actor-Critic Reinforcement Learning

1 code implementation7 May 2020 Frank Röder, Manfred Eppe, Phuong D. H. Nguyen, Stefan Wermter

Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity.

Benchmarking Hierarchical Reinforcement Learning +2

Explainable Goal-Driven Agents and Robots -- A Comprehensive Review

no code implementations21 Apr 2020 Fatai Sado, Chu Kiong Loo, Wei Shiung Liew, Matthias Kerzel, Stefan Wermter

The recent stance on the explainability of AI systems has witnessed several approaches on eXplainable Artificial Intelligence (XAI); however, most of the studies have focused on data-driven XAI systems applied in computational sciences.

Continual Learning Explainable artificial intelligence +2

Improving Robot Dual-System Motor Learning with Intrinsically Motivated Meta-Control and Latent-Space Experience Imagination

1 code implementation19 Apr 2020 Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter

In this paper, we present a novel dual-system motor learning approach where a meta-controller arbitrates online between model-based and model-free decisions based on an estimate of the local reliability of the learned model.

Robotic Grasping

Improved Techniques for Training Single-Image GANs

3 code implementations25 Mar 2020 Tobias Hinz, Matthew Fisher, Oliver Wang, Stefan Wermter

Recently there has been an interest in the potential of learning generative models from a single image, as opposed to from a large dataset.

Image Generation single-image-generation

Efficient Facial Feature Learning with Wide Ensemble-based Convolutional Neural Networks

1 code implementation17 Jan 2020 Henrique Siqueira, Sven Magg, Stefan Wermter

Experiments on large-scale datasets suggest that ESRs reduce the remaining residual generalization error on the AffectNet and FER+ datasets, reach human-level performance, and outperform state-of-the-art methods on facial expression recognition in the wild using emotion and affect concepts.

Ranked #12 on Facial Expression Recognition (FER) on FER+ (using extra training data)

Facial Expression Recognition Facial Expression Recognition (FER)

Solving Visual Object Ambiguities when Pointing: An Unsupervised Learning Approach

1 code implementation13 Dec 2019 Doreen Jirak, David Biertimpel, Matthias Kerzel, Stefan Wermter

The implementation of an intuitive gesture scenario is still challenging because both the pointing intention and the corresponding object have to be correctly recognized in real-time.

Object object-detection +1

Semantic Object Accuracy for Generative Text-to-Image Synthesis

2 code implementations29 Oct 2019 Tobias Hinz, Stefan Heinrich, Stefan Wermter

To address these challenges we introduce a new model that explicitly models individual objects within an image and a new evaluation metric called Semantic Object Accuracy (SOA) that specifically evaluates images given an image caption.

Image Captioning Text-to-Image Generation

Hierarchical Control for Bipedal Locomotion using Central Pattern Generators and Neural Networks

1 code implementation2 Sep 2019 Sayantan Auddy, Sven Magg, Stefan Wermter

Artificial central pattern generators (CPGs) can produce synchronized joint movements and have been used in the past for bipedal locomotion.

The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling

no code implementations30 Aug 2019 Pablo Barros, Nikhil Churamani, Angelica Lim, Stefan Wermter

In this paper, we propose a novel dataset composed of dyadic interactions designed, collected and annotated with a focus on measuring the affective impact that eight different stories have on the listener.

Evaluating Defensive Distillation For Defending Text Processing Neural Networks Against Adversarial Examples

1 code implementation21 Aug 2019 Marcus Soll, Tobias Hinz, Sven Magg, Stefan Wermter

Adversarial examples are artificially modified input samples which lead to misclassifications, while not being detectable by humans.

Adversarial Text General Classification +3

Towards Learning How to Properly Play UNO with the iCub Robot

1 code implementation2 Aug 2019 Pablo Barros, Stefan Wermter, Alessandra Sciutti

While interacting with another person, our reactions and behavior are much affected by the emotional changes within the temporal context of the interaction.

MoonGrad at SemEval-2019 Task 3: Ensemble BiRNNs for Contextual Emotion Detection in Dialogues

no code implementations SEMEVAL 2019 Ch Bothe, rakant, Stefan Wermter

When reading {``}I don{'}t want to talk to you any more{''}, we might interpret this as either an angry or a sad emotion in the absence of context.

From semantics to execution: Integrating action planning with reinforcement learning for robotic causal problem-solving

no code implementations23 May 2019 Manfred Eppe, Phuong D. H. Nguyen, Stefan Wermter

In this article, we build on these novel methods to facilitate the integration of action planning with reinforcement learning by exploiting the reward-sparsity as a bridge between the high-level and low-level state- and control spaces.

reinforcement-learning Reinforcement Learning (RL)

Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning

no code implementations5 May 2019 Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter

Recent success in deep reinforcement learning for continuous control has been dominated by model-free approaches which, unlike model-based approaches, do not suffer from representational limitations in making assumptions about the world dynamics and model errors inevitable in complex domains.

Continuous Control

A Personalized Affective Memory Neural Model for Improving Emotion Recognition

no code implementations23 Apr 2019 Pablo Barros, German I. Parisi, Stefan Wermter

Recent models of emotion recognition strongly rely on supervised deep learning solutions for the distinction of general emotion expressions.

Emotion Recognition

Improving interactive reinforcement learning: What makes a good teacher?

no code implementations15 Apr 2019 Francisco Cruz, Sven Magg, Yukie Nagai, Stefan Wermter

Interactive reinforcement learning has become an important apprenticeship approach to speed up convergence in classic reinforcement learning problems.

reinforcement-learning Reinforcement Learning (RL)

KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos

1 code implementation EMNLP 2018 Egor Lakomkin, Sven Magg, Cornelius Weber, Stefan Wermter

In this paper, we describe KT-Speech-Crawler: an approach for automatic dataset construction for speech recognition by crawling YouTube videos.

speech-recognition Speech Recognition

Incorporating End-to-End Speech Recognition Models for Sentiment Analysis

no code implementations28 Feb 2019 Egor Lakomkin, Mohammad Ali Zamani, Cornelius Weber, Sven Magg, Stefan Wermter

We argue that using ground-truth transcriptions during training and evaluation phases leads to a significant discrepancy in performance compared to real-world conditions, as the spoken text has to be recognized on the fly and can contain speech recognition mistakes.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Generating Multiple Objects at Spatially Distinct Locations

1 code implementation ICLR 2019 Tobias Hinz, Stefan Heinrich, Stefan Wermter

Our experiments show that through the use of the object pathway we can control object locations within images and can model complex scenes with multiple objects at various locations.

Conditional Image Generation Object +1

On the role of neurogenesis in overcoming catastrophic forgetting

no code implementations6 Nov 2018 German I. Parisi, Xu Ji, Stefan Wermter

Lifelong learning capabilities are crucial for artificial autonomous agents operating on real-world data, which is typically non-stationary and temporally correlated.

Incremental Learning

Deep Intrinsically Motivated Continuous Actor-Critic for Efficient Robotic Visuomotor Skill Learning

no code implementations26 Oct 2018 Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter

In this paper, we present a new intrinsically motivated actor-critic algorithm for learning continuous motor skills directly from raw visual input.

Continuous Control

Assessing the Contribution of Semantic Congruency to Multisensory Integration and Conflict Resolution

no code implementations15 Oct 2018 Di Fu, Pablo Barros, German I. Parisi, Haiyan Wu, Sven Magg, Xun Liu, Stefan Wermter

The efficient integration of multisensory observations is a key property of the brain that yields the robust interaction with the environment.

Unsupervised Expectation Learning for Multisensory Binding

no code implementations27 Sep 2018 Pablo Barros, German I. Parisi, Manfred Eppe, Stefan Wermter

The model adapts concepts of expectation learning to enhance the unisensory representation based on the learned bindings.

Towards Dialogue-based Navigation with Multivariate Adaptation driven by Intention and Politeness for Social Robots

no code implementations19 Sep 2018 Chandrakant Bothe, Fernando Garcia, Arturo Cruz Maya, Amit Kumar Pandey, Stefan Wermter

Service robots need to show appropriate social behaviour in order to be deployed in social environments such as healthcare, education, retail, etc.

Navigate

Curriculum goal masking for continuous deep reinforcement learning

no code implementations17 Sep 2018 Manfred Eppe, Sven Magg, Stefan Wermter

Deep reinforcement learning has recently gained a focus on problems where policy or value functions are independent of goals.

reinforcement-learning Reinforcement Learning (RL)

A Deep Neural Model Of Emotion Appraisal

2 code implementations1 Aug 2018 Pablo Barros, Emilia Barakova, Stefan Wermter

We evaluate the performance of the proposed model with different challenging corpora and compare it with state-of-the-art models for external emotion appraisal.

Developmental Learning

Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning

no code implementations26 Jul 2018 Francisco Cruz, German I. Parisi, Stefan Wermter

Additionally, we modulate the influence of sensory-driven feedback in the IRL task using goal-oriented knowledge in terms of contextual affordances.

reinforcement-learning Reinforcement Learning (RL)

Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

no code implementations19 Jul 2018 Tobias Hinz, Nicolás Navarro-Guerrero, Sven Magg, Stefan Wermter

This is independent of the underlying optimization procedure, making the approach promising for many existing hyperparameter optimization algorithms.

Hyperparameter Optimization SMAC+

Towards Modeling the Interaction of Spatial-Associative Neural Network Representations for Multisensory Perception

no code implementations13 Jul 2018 German I. Parisi, Jonathan Tong, Pablo Barros, Brigitte Röder, Stefan Wermter

In the associative layer, congruent audiovisual representations are obtained via the experience-driven development of feature-based associations.

Causal Inference

Discourse-Wizard: Discovering Deep Discourse Structure in your Conversation with RNNs

1 code implementation29 Jun 2018 Chandrakant Bothe, Sven Magg, Cornelius Weber, Stefan Wermter

Spoken language understanding is one of the key factors in a dialogue system, and a context in a conversation plays an important role to understand the current utterance.

Spoken Language Understanding

Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization

1 code implementation28 May 2018 German I. Parisi, Jun Tani, Cornelius Weber, Stefan Wermter

Both growing networks can expand in response to novel sensory experience: the episodic memory learns fine-grained spatiotemporal representations of object instances in an unsupervised fashion while the semantic memory uses task-relevant signals to regulate structural plasticity levels and develop more compact representations from episodic experience.

Active Learning Continuous Object Recognition +1

On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks

no code implementations6 Apr 2018 Egor Lakomkin, Mohammad Ali Zamani, Cornelius Weber, Sven Magg, Stefan Wermter

Speech emotion recognition (SER) is an important aspect of effective human-robot collaboration and received a lot of attention from the research community.

Data Augmentation Speech Emotion Recognition

Reusing Neural Speech Representations for Auditory Emotion Recognition

no code implementations IJCNLP 2017 Egor Lakomkin, Cornelius Weber, Sven Magg, Stefan Wermter

Acoustic emotion recognition aims to categorize the affective state of the speaker and is still a difficult task for machine learning models.

Emotion Recognition General Classification +1

GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection

no code implementations30 Mar 2018 Egor Lakomkin, Chandrakant Bothe, Stefan Wermter

Given the text of a tweet and its emotion category (anger, joy, fear, and sadness), the participants were asked to build a system that assigns emotion intensity values.

Image Generation and Translation with Disentangled Representations

no code implementations28 Mar 2018 Tobias Hinz, Stefan Wermter

We train an encoder to encode images into these representations and use a small amount of labeled data to specify what kind of information should be encoded in the disentangled part.

Conditional Image Generation Face Generation +3

The OMG-Emotion Behavior Dataset

no code implementations14 Mar 2018 Pablo Barros, Nikhil Churamani, Egor Lakomkin, Henrique Siqueira, Alexander Sutherland, Stefan Wermter

This paper is the basis paper for the accepted IJCNN challenge One-Minute Gradual-Emotion Recognition (OMG-Emotion) by which we hope to foster long-emotion classification using neural models for the benefit of the IJCNN community.

Human-Computer Interaction

Inferencing Based on Unsupervised Learning of Disentangled Representations

2 code implementations7 Mar 2018 Tobias Hinz, Stefan Wermter

Combining Generative Adversarial Networks (GANs) with encoders that learn to encode data points has shown promising results in learning data representations in an unsupervised way.

Descriptive Representation Learning +2

Continual Lifelong Learning with Neural Networks: A Review

no code implementations21 Feb 2018 German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, Stefan Wermter

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan.

Retrieval Transfer Learning

Closing the loop on multisensory interactions: A neural architecture for multisensory causal inference and recalibration

no code implementations19 Feb 2018 Jonathan Tong, German I. Parisi, Stefan Wermter, Brigitte Röder

Furthermore, we propose that these unisensory and multisensory neurons play dual roles in i) encoding spatial location as separate or integrated estimates and ii) accumulating evidence for the independence or relatedness of multisensory stimuli.

Causal Inference

Expectation Learning for Adaptive Crossmodal Stimuli Association

no code implementations23 Jan 2018 Pablo Barros, German I. Parisi, Di Fu, Xun Liu, Stefan Wermter

The human brain is able to learn, generalize, and predict crossmodal stimuli.

An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction

no code implementations22 Dec 2017 Luiza Mici, German I. Parisi, Stefan Wermter

During visuomotor tasks, robots must compensate for temporal delays inherent in their sensorimotor processing systems.

A self-organizing neural network architecture for learning human-object interactions

no code implementations5 Oct 2017 Luiza Mici, German I. Parisi, Stefan Wermter

We show that our unsupervised model shows competitive classification results on the benchmark dataset with respect to strictly supervised approaches.

Human-Object Interaction Detection Object

GradAscent at EmoInt-2017: Character and Word Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection

no code implementations WS 2017 Egor Lakomkin, Ch Bothe, rakant, Stefan Wermter

Given the text of a tweet and its emotion category (anger, joy, fear, and sadness), the participants were asked to build a system that assigns emotion intensity values.

Language Modelling Machine Translation +2

Semi-Supervised Phoneme Recognition with Recurrent Ladder Networks

no code implementations7 Jun 2017 Marian Tietz, Tayfun Alpay, Johannes Twiefel, Stefan Wermter

Ladder networks are a notable new concept in the field of semi-supervised learning by showing state-of-the-art results in image recognition tasks while being compatible with many existing neural architectures.

Interactive Natural Language Acquisition in a Multi-modal Recurrent Neural Architecture

no code implementations24 Mar 2017 Stefan Heinrich, Stefan Wermter

For the complex human brain that enables us to communicate in natural language, we gathered good understandings of principles underlying language acquisition and processing, knowledge about socio-cultural conditions, and insights about activity patterns in the brain.

Language Acquisition

Cannot find the paper you are looking for? You can Submit a new open access paper.