Search Results for author: Yanchao Yang

Found 27 papers, 12 papers with code

Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning

1 code implementation20 Sep 2023 Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, Tao Yu

Unlike inverse RL and recent work that uses LLMs to write sparse reward codes, Text2Reward produces interpretable, free-form dense reward codes that cover a wide range of tasks, utilize existing packages, and allow iterative refinement with human feedback.

reinforcement-learning Reinforcement Learning (RL)

JacobiNeRF: NeRF Shaping with Mutual Information Gradients

1 code implementation CVPR 2023 Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas

We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.

Instance Segmentation Semantic Segmentation

VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalization

1 code implementation CVPR 2023 Bingfan Zhu, Yanchao Yang, Xulong Wang, Youyi Zheng, Leonidas Guibas

We propose VDN-NeRF, a method to train neural radiance fields (NeRFs) for better geometry under non-Lambertian surface and dynamic lighting conditions that cause significant variation in the radiance of a point when viewed from different angles.

COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos

no code implementations ICCV 2023 Boxiao Pan, Bokui Shen, Davis Rempe, Despoina Paschalidou, Kaichun Mo, Yanchao Yang, Leonidas J. Guibas

In this work, we introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras.

Collision Avoidance Synthetic Data Generation

6D Camera Relocalization in Visually Ambiguous Extreme Environments

no code implementations13 Jul 2022 Yang Zheng, Tolga Birdal, Fei Xia, Yanchao Yang, Yueqi Duan, Leonidas J. Guibas

To this end, we propose: (i) a hierarchical localization system, where we leverage temporal information and (ii) a novel environment-aware image enhancement method to boost the robustness and accuracy.

Camera Relocalization Image Enhancement

SpOT: Spatiotemporal Modeling for 3D Object Tracking

no code implementations12 Jul 2022 Colton Stearns, Davis Rempe, Jie Li, Rares Ambrus, Sergey Zakharov, Vitor Guizilini, Yanchao Yang, Leonidas J Guibas

In this work, we develop a holistic representation of traffic scenes that leverages both spatial and temporal information of the actors in the scene.

3D Multi-Object Tracking 3D Object Tracking

GIMO: Gaze-Informed Human Motion Prediction in Context

1 code implementation20 Apr 2022 Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C. Karen Liu, Leonidas J. Guibas

We perform an extensive study of the benefits of leveraging the eye gaze for ego-centric human motion prediction with various state-of-the-art architectures.

Human motion prediction motion prediction

ADeLA: Automatic Dense Labeling With Attention for Viewpoint Shift in Semantic Segmentation

no code implementations CVPR 2022 Hanxiang Ren, Yanchao Yang, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas J. Guibas

We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.

Semantic Segmentation Unsupervised Domain Adaptation

Domain Adaptation on Point Clouds via Geometry-Aware Implicits

1 code implementation CVPR 2022 Yuefan Shen, Yanchao Yang, Mi Yan, He Wang, Youyi Zheng, Leonidas Guibas

Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.

Autonomous Driving Unsupervised Domain Adaptation

Object Pursuit: Building a Space of Objects via Discriminative Weight Generation

no code implementations ICLR 2022 Chuanyu Pan, Yanchao Yang, Kaichun Mo, Yueqi Duan, Leonidas Guibas

We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.


IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

no code implementations ICLR 2022 Qi Li, Kaichun Mo, Yanchao Yang, Hang Zhao, Leonidas Guibas

While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

1 code implementation29 Jul 2021 Yanchao Yang, Hanxiang Ren, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas

Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.

Inductive Bias Semantic Segmentation +1

DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis

2 code implementations27 Jul 2021 Yuefan Shen, Yanchao Yang, Youyi Zheng, C. Karen Liu, Leonidas Guibas

We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth.

Contrastive Learning Image Generation

Learning Semantic-Aware Dynamics for Video Prediction

no code implementations CVPR 2021 Xinzhu Bei, Yanchao Yang, Stefano Soatto

The appearance of the scene is warped from past frames using the predicted motion in co-visible regions; dis-occluded regions are synthesized with content-aware inpainting utilizing the predicted scene layout.

Optical Flow Estimation Video Prediction

FDA: Fourier Domain Adaptation for Semantic Segmentation

3 code implementations CVPR 2020 Yanchao Yang, Stefano Soatto

We describe a simple method for unsupervised domain adaptation, whereby the discrepancy between the source and target distributions is reduced by swapping the low-frequency spectrum of one with the other.

Segmentation Semantic Segmentation +1

Learning to Manipulate Individual Objects in an Image

1 code implementation CVPR 2020 Yanchao Yang, Yutong Chen, Stefano Soatto

We describe a method to train a generative model with latent factors that are (approximately) independent and localized.


Phase Consistent Ecological Domain Adaptation

1 code implementation CVPR 2020 Yanchao Yang, Dong Lao, Ganesh Sundaramoorthi, Stefano Soatto

We introduce two criteria to regularize the optimization involved in learning a classifier in a domain where no annotated data are available, leveraging annotated data in a different domain, a problem known as unsupervised domain adaptation.

Segmentation Semantic Segmentation +1

Dense Depth Posterior (DDP) from Single Image and Sparse Range

no code implementations CVPR 2019 Yanchao Yang, Alex Wong, Stefano Soatto

We present a deep learning system to infer the posterior distribution of a dense depth map associated with an image, by exploiting sparse range measurements, for instance from a lidar.

Depth Completion Test

Conditional Prior Networks for Optical Flow

1 code implementation ECCV 2018 Yanchao Yang, Stefano Soatto

On the other hand, fully supervised methods learn the regularity in the annotated data, without explicit regularization and with the risk of overfitting.

Optical Flow Estimation

S2F: Slow-To-Fast Interpolator Flow

no code implementations CVPR 2017 Yanchao Yang, Stefano Soatto

We introduce a method to compute optical flow at multiple scales of motion, without resorting to multi- resolution or combinatorial methods.

Optical Flow Estimation

Self-Occlusions and Disocclusions in Causal Video Object Segmentation

no code implementations ICCV 2015 Yanchao Yang, Ganesh Sundaramoorthi, Stefano Soatto

We propose a method to detect disocclusion in video sequences of three-dimensional scenes and to partition the disoccluded regions into objects, defined by coherent deformation corresponding to surfaces in the scene.

Semantic Segmentation Video Object Segmentation +1

Shape Tracking With Occlusions via Coarse-To-Fine Region-Based Sobolev Descent

no code implementations21 Aug 2012 Yanchao Yang, Ganesh Sundaramoorthi

In cases of 3D object motion and viewpoint change, self-occlusions and dis-occlusions of the object are prominent, and current methods employing joint shape and appearance models are unable to adapt to new shape and appearance information, leading to inaccurate shape detection.

Object Tracking

Cannot find the paper you are looking for? You can Submit a new open access paper.