Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection

no code implementations • 17 Dec 2023 • Xinghao Zhu, Devesh K. Jha, Diego Romeres, Lingfeng Sun, Masayoshi Tomizuka, Anoop Cherian

Automating the assembly of objects from their parts is a complex problem with innumerable applications in manufacturing, maintenance, and recycling.

Motion Planning valid

Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks

no code implementations • 11 Dec 2023 • Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddarth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka, Diego Romeres

Designing robotic agents to perform open vocabulary tasks has been the long-standing goal in robotics and AI.

Human-oriented Representation Learning for Robotic Manipulation

no code implementations • 4 Oct 2023 • Mingxiao Huo, Mingyu Ding, Chenfeng Xu, Thomas Tian, Xinghao Zhu, Yao Mu, Lingfeng Sun, Masayoshi Tomizuka, Wei Zhan

We introduce Task Fusion Decoder as a plug-and-play embedding translator that utilizes the underlying relationships among these perceptual skills to guide the representation learning towards encoding meaningful structure for what's important for all perceptual skills, ultimately empowering learning of downstream robotic manipulation tasks.

Hand Detection Representation Learning +1

Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework

no code implementations • 2 Jun 2023 • Lingfeng Sun, Haichao Zhang, Wei Xu, Masayoshi Tomizuka

In this work, we investigate the potential of improving multi-task training and also leveraging it for transferring in the reinforcement learning setting.

reinforcement-learning Transfer Reinforcement Learning

PaCo: Parameter-Compositional Multi-Task Reinforcement Learning

1 code implementation • 21 Oct 2022 • Lingfeng Sun, Haichao Zhang, Wei Xu, Masayoshi Tomizuka

However, the gaps between contents and difficulties of different tasks bring us challenges on both which tasks should share the parameters and what parameters should be shared, as well as the optimization challenges due to parameter sharing.

reinforcement-learning Reinforcement Learning (RL)

PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Map

1 code implementation • 21 Apr 2022 • Chenfeng Xu, Tian Li, Chen Tang, Lingfeng Sun, Kurt Keutzer, Masayoshi Tomizuka, Alireza Fathi, Wei Zhan

It is hard to replicate these approaches in trajectory forecasting due to the lack of adequate trajectory data (e. g., 34K samples in the nuScenes dataset).

Contrastive Learning Representation Learning +1

Domain Knowledge Driven Pseudo Labels for Interpretable Goal-Conditioned Interactive Trajectory Prediction

no code implementations • 28 Mar 2022 • Lingfeng Sun, Chen Tang, Yaru Niu, Enna Sachdeva, Chiho Choi, Teruhisa Misu, Masayoshi Tomizuka, Wei Zhan

To address these issues, we propose a novel approach to avoid KL vanishing and induce an interpretable interactive latent space with pseudo labels.

Motion Forecasting Trajectory Prediction

Cross Domain Robot Imitation with Invariant Representation

1 code implementation • 13 Sep 2021 • Zhao-Heng Yin, Lingfeng Sun, Hengbo Ma, Masayoshi Tomizuka, Wu-Jun Li

In this paper, we consider CDIL on a class of similar robots.

Imitation Learning Representation Learning

Diverse Critical Interaction Generation for Planning and Planner Evaluation

no code implementations • 1 Mar 2021 • Zhao-Heng Yin, Lingfeng Sun, Liting Sun, Masayoshi Tomizuka, Wei Zhan

Experiments show that our model can generate diverse interactions in various scenarios.

Autonomous Vehicles Decision Making