Search Results for author:
Found 68 papers, 21 papers with code
Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision
We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework.
Can Transformers Smell Like Humans?
The human brain encodes stimuli from the environment into representations that form a sensory perception of the world.
A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics
By incorporating physical consistency as inductive bias, deep neural networks display increased generalization capabilities and data efficiency in learning nonlinear dynamic models.
Reducing Variance in Meta-Learning via Laplace Approximation for Regression Tasks
Given a finite set of sample points, meta-learning algorithms aim to learn an optimal adaptation strategy for new, unseen tasks.
Relative Representations: Topological and Geometric Perspectives
Based on insights of topological and geometric nature, we propose two improvements to relative representations.
Hyperbolic Delaunay Geometric Alignment
Hyperbolic machine learning is an emerging field aimed at representing data with a hierarchical structure.
LM2D: Lyrics- and Music-Driven Dance Synthesis
First, we propose LM2D, a novel probabilistic architecture that incorporates a multimodal diffusion model with consistency distillation, designed to create dance conditioned on both music and lyrics in one diffusion generation step.
Learning Goal-Conditioned Policies from Sub-Optimal Offline Data via Metric Learning
We use the proposed value function to guide the learning of a policy in an actor-critic fashion, a method we name MetricRL.
The Essential Role of Causality in Foundation World Models for Embodied AI
The study of causality lends itself to the construction of veridical world models, which are crucial for accurately predicting the outcomes of possible interactions.
DexDiffuser: Generating Dexterous Grasps with Diffusion Models
We introduce DexDiffuser, a novel dexterous grasping method that generates, evaluates, and refines grasps on partial object point clouds.
Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks
In this work, we formally prove that, under certain conditions, if a neural network is invariant to a finite group then its weights recover the Fourier transform on that group.
Scalable Motion Style Transfer with Constrained Diffusion Generation
Current training of motion style transfer systems relies on consistency losses across style domains to preserve contents, hindering its scalable application to a large number of domains and private data.
Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges
Transfer learning is a conceptually-enticing paradigm in pursuit of truly intelligent embodied agents.
Music- and Lyrics-driven Dance Synthesis
To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics.
Learning Geometric Representations of Objects via Interaction
We address the problem of learning representations from observations of a scene involving an agent and an external object the agent interacts with.
TD-GEM: Text-Driven Garment Editing Mapper
An optimization-based Contrastive Language-Image Pre-training (CLIP) is then utilized to guide the latent representation of a fashion image in the direction of a target attribute expressed in terms of a text prompt.
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models
Data-driven and controllable human motion synthesis and prediction are active research areas with various applications in interactive media and social robotics.
Equivariant Representation Learning in the Presence of Stabilizers
We introduce Equivariant Isomorphic Networks (EquIN) -- a method for learning representations that are equivariant with respect to general group actions over data.
EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
We study the problem of learning graph dynamics of deformable objects that generalizes to unknown physical properties.
Dance Style Transfer with Cross-modal Transformer
We present CycleDance, a dance style transfer system to transform an existing motion clip in one dance style to a motion clip in another dance style while attempting to preserve motion context of the dance.
Digraphwave: Scalable Extraction of Structural Node Embeddings via Diffusion on Directed Graphs
Structural node embeddings, vectors capturing local connectivity information for each node in a graph, have many applications in data mining and machine learning, e. g., network alignment and node classification, clustering and anomaly detection.
Back to the Manifold: Recovering from Out-of-Distribution States
However, a major challenge is a distributional shift between the states in the training dataset and the ones visited by the learned policy at the test time.
On the Subspace Structure of Gradient-Based Meta-Learning
In this work we provide an analysis of the distribution of the post-adaptation parameters of Gradient-Based Meta-Learning (GBML) methods.
Equivariant Representation Learning via Class-Pose Decomposition
We introduce a general method for learning representations that are equivariant to symmetries of data.
Active Nearest Neighbor Regression Through Delaunay Refinement
We introduce an algorithm for active function approximation based on nearest neighbor regression.
Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models
We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models.
Delaunay Component Analysis for Evaluation of Data Representations
Advanced representation learning techniques require reliable and general evaluation methods.
GraphDCA -- a Framework for Node Distribution Comparison in Real and Synthetic Graphs
We argue that when comparing two graphs, the distribution of node structural features is more informative than global graph statistics which are often used in practice, especially to evaluate graph generative models.
Geometric Multimodal Contrastive Representation Learning
Learning representations of multimodal data that are both informative and robust to missing modalities at test time remains a challenging problem due to the inherent heterogeneity of data obtained from different channels.
Comparing Reconstruction- and Contrastive-based Models for Visual Task Planning
Most methods learn state representations by utilizing losses based on the reconstruction of the raw observations from a lower-dimensional latent space.
Batch Curation for Unsupervised Contrastive Representation Learning
The state-of-the-art unsupervised contrastive visual representation learning methods that have emerged recently (SimCLR, MoCo, SwAV) all make use of data augmentations in order to construct a pretext task of instant discrimination consisting of similar and dissimilar pairs of images.
GeomCA: Geometric Evaluation of Data Representations
Evaluating the quality of learned representations without relying on a downstream task remains one of the challenges in representation learning.
Graph-based Normalizing Flow for Human Motion Generation and Reconstruction
Data-driven approaches for modeling human skeletal motion have found various applications in interactive media and social robotics.
FEW-SHOTLEARNING WITH WEAK SUPERVISION
Few-shot meta-learning methods aim to learn the common structure shared across a set of tasks to facilitate learning new tasks with small amounts of data.
Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms
Our results show that the proposed method can successfully adapt a trained policy to different robotic platforms with novel physical parameters and the superiority of our meta-learning algorithm compared to state-of-the-art methods for the introduced few-shot policy adaptation problem.
Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects
In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks.
Robotics
Enabling Visual Action Planning for Object Manipulation through Latent Space Roadmap
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces, focusing on manipulation of deformable objects.
Interpretability in Contact-Rich Manipulation via Kinodynamic Images
In this work, we address the interpretability of NN-based models by introducing the kinodynamic images.
Robotics
Sequential Topological Representations for Predictive Models of Deformable Objects
Deformable objects present a formidable challenge for robotic manipulation due to the lack of canonical low-dimensional representations and the difficulty of capturing, predicting, and controlling such objects.
Witness Autoencoder: Shaping the Latent Space with Witness Complexes
We present a Witness Autoencoder (W-AE) – an autoencoder that captures geodesic distances of the data in the latent space.
Data-efficient visuomotor policy training using reinforcement learning and generative models
We present a data-efficient framework for solving visuomotor sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models.
Human-centered collaborative robots with deep reinforcement learning
We present a reinforcement learning based framework for human-centered collaborative systems.
Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control
Our second contribution is a unifying mathematical formulation for learning latent relations.
The effect of Target Normalization and Momentum on Dying ReLU
To further investigate this matter, we analyze a discrete-time linear autonomous system, and show theoretically how this relates to a model with a single ReLU and how common properties can result in dying ReLU.
Fashion Landmark Detection and Category Classification for Robotics
Research on automated, image based identification of clothing categories and fashion landmarks has recently gained significant interest due to its potential impact on areas such as robotic clothing manipulation, automated clothes sorting and recycling, and online shopping.
Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces such as manipulation of deformable objects.
Benchmarking In-Hand Manipulation
The purpose of this benchmark is to evaluate the planning and control aspects of robotic in-hand manipulation systems.
Robotics
Learning Task-Oriented Grasping from Human Activity Datasets
We develop a model that takes as input an RGB image and outputs a hand pose and configuration as well as an object pose and a shape.
Seeing the whole picture instead of a single point: Self-supervised likelihood learning for deep generative models
Recent findings show that deep generative models can judge out-of-distribution samples as more likely than those drawn from the same distribution as the training data.
Imitating by generating: deep generative models for imitation of interactive tasks
To coordinate actions with an interaction partner requires a constant exchange of sensorimotor signals.
Probabilistic Model Learning and Long-term Prediction for Contact-rich Manipulation Tasks
Learning dynamics models is an essential component of model-based reinforcement learning.
Robotics
Fast Adaptation with Meta-Reinforcement Learning for Trust Modelling in Human-Robot Interaction
In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction.
Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
We propose a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way.
Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images
The qualitative experiments show results of pose and shape estimation of objects held by a hand "in the wild".
Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
Our further contribution is a neural network architecture and training pipeline that use experience from grasping objects in simulation to learn grasp stability scores.
A Probabilistic Semi-Supervised Approach to Multi-Task Human Activity Modeling
Therefore, video-based human activity modeling is concerned with a number of tasks such as inferring current and future semantic labels, predicting future continuous observations as well as imagining possible future label and feature sequences.
Detect, anticipate and generate: Semi-supervised recurrent latent variable models for human activity modeling
In this work we introduce semi-supervised variational recurrent neural networks which are able to a) model temporal distributions over latent factors and the observable feature space, b) incorporate discrete labels such as activity type when available, and c) generate possible future action sequences on both feature and label level.
Reinforcement Learning in Topology-based Representation for Human Body Movement with Whole Arm Manipulation
Moving a human body or a large and bulky object can require the strength of whole arm manipulation (WAM).
VPE: Variational Policy Embedding for Transfer Reinforcement Learning
The low-dimensional space, and master policy found by our method enables policies to quickly adapt to new environments.
Rearrangement with Nonprehensile Manipulation Using Deep Reinforcement Learning
Usually, this is achieved by precisely modeling physical properties of the objects, robot, and the environment for explicit planning.
Reinforcement Learning for Pivoting Task
In this work we propose an approach to learn a robust policy for solving the pivoting task.
Anticipating many futures: Online human motion prediction and synthesis for human-robot collaboration
Fluent and safe interactions of humans and robots require both partners to anticipate the others' actions.
Deep representation learning for human motion prediction and classification
To quantify the learned features, we use the output of different layers for action classification and visualize the receptive fields of the network units.
A Sensorimotor Reinforcement Learning Framework for Physical Human-Robot Interaction
Modeling of physical human-robot collaborations is generally a challenging problem due to the unpredictive nature of human behavior.
Feature Descriptors for Tracking by Detection: a Benchmark
In this paper, we provide an extensive evaluation of the performance of local descriptors for tracking applications.
Interactive Perception: Leveraging Action in Perception and Perception in Action
Recent approaches in robotics follow the insight that perception is facilitated by interaction with the environment.
Robotics
Data-Driven Grasp Synthesis - A Survey
In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation.
Robotics
Persistent Homology for Learning Densities with Bounded Support
We present a novel method for learning densities with bounded support which enables us to incorporate `hard' topological constraints.
