Search Results for author: Kuang-Huei Lee

Found 17 papers, 8 papers with code

Language to Rewards for Robotic Skill Synthesis

no code implementations14 Jun 2023 Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, Fei Xia

However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot.

Logical Reasoning

Multimodal Web Navigation with Instruction-Finetuned Foundation Models

no code implementations19 May 2023 Hiroki Furuta, Ofir Nachum, Kuang-Huei Lee, Yutaka Matsuo, Shixiang Shane Gu, Izzeddin Gur

The progress of autonomous web navigation has been hindered by the dependence on billions of exploratory interactions via online reinforcement learning, and domain-specific model designs that make it difficult to leverage generalization from rich out-of-domain data.

Instruction Following Language Modelling

Open-World Object Manipulation using Pre-trained Vision-Language Models

no code implementations2 Mar 2023 Austin Stone, Ted Xiao, Yao Lu, Keerthana Gopalakrishnan, Kuang-Huei Lee, Quan Vuong, Paul Wohlhart, Brianna Zitkovich, Fei Xia, Chelsea Finn, Karol Hausman

This brings up a notably difficult challenge for robots: while robot learning approaches allow robots to learn many different behaviors from first-hand experience, it is impractical for robots to have first-hand experiences that span all of this semantic information.

Language Modelling

PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale

no code implementations15 Oct 2022 Kuang-Huei Lee, Ted Xiao, Adrian Li, Paul Wohlhart, Ian Fischer, Yao Lu

The predictive information, the mutual information between the past and future, has been shown to be a useful representation learning auxiliary loss for training reinforcement learning agents, as the ability to model what will happen next is critical to success on many control tasks.

reinforcement-learning Reinforcement Learning (RL) +2

PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations

no code implementations27 Jul 2022 Kuang-Huei Lee, Ofir Nachum, Tingnan Zhang, Sergio Guadarrama, Jie Tan, Wenhao Yu

Evolution Strategy (ES) algorithms have shown promising results in training complex robotic control policies due to their massive parallelism capability, simple implementation, effective parameter-space exploration, and fast training time.

Representation Learning

Deep Hierarchical Planning from Pixels

no code implementations8 Jun 2022 Danijar Hafner, Kuang-Huei Lee, Ian Fischer, Pieter Abbeel

Despite operating in latent space, the decisions are interpretable because the world model can decode goals into images for visualization.

Atari Games Hierarchical Reinforcement Learning

Multi-Game Decision Transformers

1 code implementation30 May 2022 Kuang-Huei Lee, Ofir Nachum, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, Igor Mordatch

Specifically, we show that a single transformer-based model - with a single set of weights - trained purely offline can play a suite of up to 46 Atari games simultaneously at close-to-human performance.

Atari Games Offline RL

An Empirical Investigation of Representation Learning for Imitation

2 code implementations16 May 2022 Xin Chen, Sam Toyer, Cody Wild, Scott Emmons, Ian Fischer, Kuang-Huei Lee, Neel Alex, Steven H Wang, Ping Luo, Stuart Russell, Pieter Abbeel, Rohin Shah

We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation across several environment suites.

Image Classification Imitation Learning +1

Compressive Visual Representations

1 code implementation NeurIPS 2021 Kuang-Huei Lee, Anurag Arnab, Sergio Guadarrama, John Canny, Ian Fischer

We verify this by developing SimCLR and BYOL formulations compatible with the Conditional Entropy Bottleneck (CEB) objective, allowing us to both measure and control the amount of compression in the learned representation, and observe their impact on downstream tasks.

Contrastive Learning Self-Supervised Image Classification

Learning Visual Relation Priors for Image-Text Matching and Image Captioning with Neural Scene Graph Generators

no code implementations22 Sep 2019 Kuang-Huei Lee, Hamid Palangi, Xi Chen, Houdong Hu, Jianfeng Gao

In this work, we tackle two fundamental language-and-vision tasks: image-text matching and image captioning, and demonstrate that neural scene graph generators can learn effective visual relation features to facilitate grounding language to visual relations and subsequently improve the two end applications.

Image Captioning Image-text matching +1

Stacked Cross Attention for Image-Text Matching

6 code implementations ECCV 2018 Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, Xiaodong He

Prior work either simply aggregates the similarity of all possible pairs of regions and words without attending differentially to more and less important words or regions, or uses a multi-step attentional process to capture limited number of semantic alignments which is less interpretable.

Image Retrieval Image-text matching +4

CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise

2 code implementations CVPR 2018 Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang

We demonstrate the effectiveness of the proposed algorithm on both of the label noise detection task and the image classification on noisy data task on several large-scale datasets.

Ranked #2 on Image Classification on Food-101N (using extra training data)

Classification General Classification +2

Cannot find the paper you are looking for? You can Submit a new open access paper.