Search Results for author:
Found 63 papers, 19 papers with code
Persistent Homology for Learning Densities with Bounded Support
We present a novel method for learning densities with bounded support which enables us to incorporate `hard' topological constraints.
Data-Driven Grasp Synthesis - A Survey
In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation.
Robotics
Interactive Perception: Leveraging Action in Perception and Perception in Action
Recent approaches in robotics follow the insight that perception is facilitated by interaction with the environment.
Robotics
Feature Descriptors for Tracking by Detection: a Benchmark
In this paper, we provide an extensive evaluation of the performance of local descriptors for tracking applications.
A Sensorimotor Reinforcement Learning Framework for Physical Human-Robot Interaction
Modeling of physical human-robot collaborations is generally a challenging problem due to the unpredictive nature of human behavior.
Deep representation learning for human motion prediction and classification
To quantify the learned features, we use the output of different layers for action classification and visualize the receptive fields of the network units.
Anticipating many futures: Online human motion prediction and synthesis for human-robot collaboration
Fluent and safe interactions of humans and robots require both partners to anticipate the others' actions.
Reinforcement Learning for Pivoting Task
In this work we propose an approach to learn a robust policy for solving the pivoting task.
Rearrangement with Nonprehensile Manipulation Using Deep Reinforcement Learning
Usually, this is achieved by precisely modeling physical properties of the objects, robot, and the environment for explicit planning.
VPE: Variational Policy Embedding for Transfer Reinforcement Learning
The low-dimensional space, and master policy found by our method enables policies to quickly adapt to new environments.
Reinforcement Learning in Topology-based Representation for Human Body Movement with Whole Arm Manipulation
Moving a human body or a large and bulky object can require the strength of whole arm manipulation (WAM).
Detect, anticipate and generate: Semi-supervised recurrent latent variable models for human activity modeling
In this work we introduce semi-supervised variational recurrent neural networks which are able to a) model temporal distributions over latent factors and the observable feature space, b) incorporate discrete labels such as activity type when available, and c) generate possible future action sequences on both feature and label level.
A Probabilistic Semi-Supervised Approach to Multi-Task Human Activity Modeling
Therefore, video-based human activity modeling is concerned with a number of tasks such as inferring current and future semantic labels, predicting future continuous observations as well as imagining possible future label and feature sequences.
Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
Our further contribution is a neural network architecture and training pipeline that use experience from grasping objects in simulation to learn grasp stability scores.
Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images
The qualitative experiments show results of pose and shape estimation of objects held by a hand "in the wild".
Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
We propose a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way.
Fast Adaptation with Meta-Reinforcement Learning for Trust Modelling in Human-Robot Interaction
In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction.
Probabilistic Model Learning and Long-term Prediction for Contact-rich Manipulation Tasks
Learning dynamics models is an essential component of model-based reinforcement learning.
Robotics
Imitating by generating: deep generative models for imitation of interactive tasks
To coordinate actions with an interaction partner requires a constant exchange of sensorimotor signals.
Seeing the whole picture instead of a single point: Self-supervised likelihood learning for deep generative models
Recent findings show that deep generative models can judge out-of-distribution samples as more likely than those drawn from the same distribution as the training data.
Learning Task-Oriented Grasping from Human Activity Datasets
We develop a model that takes as input an RGB image and outputs a hand pose and configuration as well as an object pose and a shape.
Benchmarking In-Hand Manipulation
The purpose of this benchmark is to evaluate the planning and control aspects of robotic in-hand manipulation systems.
Robotics
Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces such as manipulation of deformable objects.
Fashion Landmark Detection and Category Classification for Robotics
Research on automated, image based identification of clothing categories and fashion landmarks has recently gained significant interest due to its potential impact on areas such as robotic clothing manipulation, automated clothes sorting and recycling, and online shopping.
The effect of Target Normalization and Momentum on Dying ReLU
To further investigate this matter, we analyze a discrete-time linear autonomous system, and show theoretically how this relates to a model with a single ReLU and how common properties can result in dying ReLU.
Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control
Our second contribution is a unifying mathematical formulation for learning latent relations.
Human-centered collaborative robots with deep reinforcement learning
We present a reinforcement learning based framework for human-centered collaborative systems.
Data-efficient visuomotor policy training using reinforcement learning and generative models
We present a data-efficient framework for solving visuomotor sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models.
Witness Autoencoder: Shaping the Latent Space with Witness Complexes
We present a Witness Autoencoder (W-AE) – an autoencoder that captures geodesic distances of the data in the latent space.
Sequential Topological Representations for Predictive Models of Deformable Objects
Deformable objects present a formidable challenge for robotic manipulation due to the lack of canonical low-dimensional representations and the difficulty of capturing, predicting, and controlling such objects.
Interpretability in Contact-Rich Manipulation via Kinodynamic Images
In this work, we address the interpretability of NN-based models by introducing the kinodynamic images.
Robotics
Enabling Visual Action Planning for Object Manipulation through Latent Space Roadmap
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces, focusing on manipulation of deformable objects.
Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects
In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks.
Robotics
Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms
Our results show that the proposed method can successfully adapt a trained policy to different robotic platforms with novel physical parameters and the superiority of our meta-learning algorithm compared to state-of-the-art methods for the introduced few-shot policy adaptation problem.
FEW-SHOTLEARNING WITH WEAK SUPERVISION
Few-shot meta-learning methods aim to learn the common structure shared across a set of tasks to facilitate learning new tasks with small amounts of data.
Graph-based Normalizing Flow for Human Motion Generation and Reconstruction
Data-driven approaches for modeling human skeletal motion have found various applications in interactive media and social robotics.
GeomCA: Geometric Evaluation of Data Representations
Evaluating the quality of learned representations without relying on a downstream task remains one of the challenges in representation learning.
Batch Curation for Unsupervised Contrastive Representation Learning
The state-of-the-art unsupervised contrastive visual representation learning methods that have emerged recently (SimCLR, MoCo, SwAV) all make use of data augmentations in order to construct a pretext task of instant discrimination consisting of similar and dissimilar pairs of images.
Comparing Reconstruction- and Contrastive-based Models for Visual Task Planning
Most methods learn state representations by utilizing losses based on the reconstruction of the raw observations from a lower-dimensional latent space.
Geometric Multimodal Contrastive Representation Learning
Learning representations of multimodal data that are both informative and robust to missing modalities at test time remains a challenging problem due to the inherent heterogeneity of data obtained from different channels.
GraphDCA -- a Framework for Node Distribution Comparison in Real and Synthetic Graphs
We argue that when comparing two graphs, the distribution of node structural features is more informative than global graph statistics which are often used in practice, especially to evaluate graph generative models.
Delaunay Component Analysis for Evaluation of Data Representations
Advanced representation learning techniques require reliable and general evaluation methods.
Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models
We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models.
Active Nearest Neighbor Regression Through Delaunay Refinement
We introduce an algorithm for active function approximation based on nearest neighbor regression.
Equivariant Representation Learning via Class-Pose Decomposition
We introduce a general method for learning representations that are equivariant to symmetries of data.
On the Subspace Structure of Gradient-Based Meta-Learning
In this work we provide an analysis of the distribution of the post-adaptation parameters of Gradient-Based Meta-Learning (GBML) methods.
Back to the Manifold: Recovering from Out-of-Distribution States
However, a major challenge is a distributional shift between the states in the training dataset and the ones visited by the learned policy at the test time.
Digraphwave: Scalable Extraction of Structural Node Embeddings via Diffusion on Directed Graphs
Structural node embeddings, vectors capturing local connectivity information for each node in a graph, have many applications in data mining and machine learning, e. g., network alignment and node classification, clustering and anomaly detection.
Dance Style Transfer with Cross-modal Transformer
We present CycleDance, a dance style transfer system to transform an existing motion clip in one dance style to a motion clip in another dance style while attempting to preserve motion context of the dance.
EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
We study the problem of learning graph dynamics of deformable objects that generalizes to unknown physical properties.
Equivariant Representation Learning in the Presence of Stabilizers
We introduce Equivariant Isomorphic Networks (EquIN) -- a method for learning representations that are equivariant with respect to general group actions over data.
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models
Data-driven and controllable human motion synthesis and prediction are active research areas with various applications in interactive media and social robotics.
TD-GEM: Text-Driven Garment Editing Mapper
An optimization-based Contrastive Language-Image Pre-training (CLIP) is then utilized to guide the latent representation of a fashion image in the direction of a target attribute expressed in terms of a text prompt.
Learning Geometric Representations of Objects via Interaction
We address the problem of learning representations from observations of a scene involving an agent and an external object the agent interacts with.
Music- and Lyrics-driven Dance Synthesis
To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics.
Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges
Transfer learning is a conceptually-enticing paradigm in pursuit of truly intelligent embodied agents.
Scalable Motion Style Transfer with Constrained Diffusion Generation
Current training of motion style transfer systems relies on consistency losses across style domains to preserve contents, hindering its scalable application to a large number of domains and private data.
Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks
In this work, we formally prove that, under certain conditions, if a neural network is invariant to a finite group then its weights recover the Fourier transform on that group.
DexDiffuser: Generating Dexterous Grasps with Diffusion Models
We introduce DexDiffuser, a novel dexterous grasping method that generates, evaluates, and refines grasps on partial object point clouds.
The Essential Role of Causality in Foundation World Models for Embodied AI
This paper focuses on the prospects of building foundation world models for the upcoming generation of embodied agents and presents a novel viewpoint on the significance of causality within these.
Goal-Conditioned Offline Reinforcement Learning via Metric Learning
Experimentally, we show how our method consistently outperforms other offline RL baselines in learning from sub-optimal offline datasets.
LM2D: Lyrics- and Music-Driven Dance Synthesis
First, we propose LM2D, a novel probabilistic architecture that incorporates a multimodal diffusion model with consistency distillation, designed to create dance conditioned on both music and lyrics in one diffusion generation step.
Hyperbolic Delaunay Geometric Alignment
Hyperbolic machine learning is an emerging field aimed at representing data with a hierarchical structure.
