Search Results for author: Hao Su

Found 141 papers, 80 papers with code

TensoRF: Tensorial Radiance Fields

2 code implementations17 Mar 2022 Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, Hao Su

We demonstrate that applying traditional CP decomposition -- that factorizes tensors into rank-one components with compact vectors -- in our framework leads to improvements over vanilla NeRF.

Low-Dose X-Ray Ct Reconstruction Novel View Synthesis

Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model

1 code implementation23 Oct 2023 Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, Hao Su

We report Zero123++, an image-conditioned diffusion model for generating 3D-consistent multi-view images from a single input view.

MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

2 code implementations ICCV 2021 Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, Hao Su

We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis.

Neural Rendering

SofGAN: A Portrait Image Generator with Dynamic Styling

1 code implementation7 Jul 2020 Anpei Chen, Ruiyang Liu, Ling Xie, Zhang Chen, Hao Su, Jingyi Yu

To address this issue, we propose a SofGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space.

2D Semantic Segmentation Image Generation +1

ImageNet Large Scale Visual Recognition Challenge

12 code implementations1 Sep 2014 Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, Li Fei-Fei

The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images.

General Classification Image Classification +4

Robust Watertight Manifold Surface Generation Method for ShapeNet Models

2 code implementations5 Feb 2018 Jingwei Huang, Hao Su, Leonidas Guibas

In this paper, we describe a robust algorithm for 2-Manifold generation of various kinds of ShapeNet Models.

Computational Geometry

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

4 code implementations CVPR 2017 Haoqiang Fan, Hao Su, Leonidas Guibas

Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image.

Ranked #2 on 3D Reconstruction on Data3D−R2N2 (using extra training data)

3D Object Reconstruction From A Single Image 3D Reconstruction

Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search

1 code implementation5 May 2022 Xinyue Wei, Minghua Liu, Zhan Ling, Hao Su

Approximate convex decomposition aims to decompose a 3D shape into a set of almost convex components, whose convex hulls can then be used to represent the input shape.

SAPIEN: A SimulAted Part-based Interactive ENvironment

1 code implementation CVPR 2020 Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su

To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.

Attribute

Temporal Difference Learning for Model Predictive Control

2 code implementations9 Mar 2022 Nicklas Hansen, Xiaolong Wang, Hao Su

Data-driven model predictive control has two key advantages over model-free methods: a potential for improved sample efficiency through model learning, and better performance as computational budget for planning increases.

Continuous Control Model Predictive Control

NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction

1 code implementation CVPR 2022 Xiaoshuai Zhang, Sai Bi, Kalyan Sunkavalli, Hao Su, Zexiang Xu

We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.

3D Reconstruction

StructureNet: Hierarchical Graph Networks for 3D Shape Generation

2 code implementations1 Aug 2019 Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas

We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families; and (iii) can be used to generate a great diversity of realistic structured shape geometries.

3D Shape Generation

GNeRF: GAN-based Neural Radiance Field without Posed Camera

1 code implementation ICCV 2021 Quan Meng, Anpei Chen, Haimin Luo, Minye Wu, Hao Su, Lan Xu, Xuming He, Jingyi Yu

We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field (NeRF) reconstruction for the complex scenarios with unknown and even randomly initialized camera poses.

Novel View Synthesis

Volumetric and Multi-View CNNs for Object Classification on 3D Data

2 code implementations CVPR 2016 Charles R. Qi, Hao Su, Matthias Niessner, Angela Dai, Mengyuan Yan, Leonidas J. Guibas

Empirical results from these two types of CNNs exhibit a large gap, indicating that existing volumetric CNN architectures and approaches are unable to fully exploit the power of 3D representations.

3D Object Recognition 3D Point Cloud Classification +1

FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization

1 code implementation2 Dec 2021 Xingchao Liu, Chengyue Gong, Lemeng Wu, Shujian Zhang, Hao Su, Qiang Liu

We approach text-to-image generation by combining the power of the retrained CLIP representation with an off-the-shelf image generator (GANs), optimizing in the latent space of GAN to find images that achieve maximum CLIP score with the given input text.

counterfactual Navigate +1

OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding

1 code implementation NeurIPS 2023 Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, Hao Su

Due to their alignment with CLIP embeddings, our learned shape representations can also be integrated with off-the-shelf CLIP-based models for various applications, such as point cloud captioning and point cloud-conditioned image generation.

3D Classification 3D Shape Representation +4

Factor Fields: A Unified Framework for Neural Fields and Beyond

1 code implementation2 Feb 2023 Anpei Chen, Zexiang Xu, Xinyue Wei, Siyu Tang, Hao Su, Andreas Geiger

Our experiments show that DiF leads to improvements in approximation quality, compactness, and training time when compared to previous fast reconstruction methods.

regression

TD-MPC2: Scalable, Robust World Models for Continuous Control

1 code implementation25 Oct 2023 Nicklas Hansen, Hao Su, Xiaolong Wang

TD-MPC is a model-based reinforcement learning (RL) algorithm that performs local trajectory optimization in the latent space of a learned implicit (decoder-free) world model.

Continuous Control Model-based Reinforcement Learning +1

Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints

1 code implementation29 Jul 2020 You-Yi Jau, Rui Zhu, Hao Su, Manmohan Chandraker

Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry (VO) and simultaneous localization and mapping (SLAM), where classic methods consisting of hand-crafted features and sampling-based outlier rejection have been a dominant choice for over a decade.

Pose Estimation Simultaneous Localization and Mapping +1

Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation

3 code implementations NeurIPS 2021 Nicklas Hansen, Hao Su, Xiaolong Wang

Our method greatly improves stability and sample efficiency of ConvNets under augmentation, and achieves generalization results competitive with state-of-the-art methods for image-based RL in environments with unseen visuals.

Data Augmentation Q-Learning +1

Learning Shape Abstractions by Assembling Volumetric Primitives

4 code implementations CVPR 2017 Shubham Tulsiani, Hao Su, Leonidas J. Guibas, Alexei A. Efros, Jitendra Malik

We present a learning framework for abstracting complex shapes by learning to assemble objects using 3D volumetric primitives.

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations

3 code implementations30 Jul 2021 Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao Su

Here we propose SAPIEN Manipulation Skill Benchmark (ManiSkill) to benchmark manipulation skills over diverse objects in a full-physics simulator.

Strivec: Sparse Tri-Vector Radiance Fields

1 code implementation ICCV 2023 Quankai Gao, Qiangeng Xu, Hao Su, Ulrich Neumann, Zexiang Xu

In contrast to TensoRF which uses a global tensor and focuses on their vector-matrix decomposition, we propose to utilize a cloud of local tensors and apply the classic CANDECOMP/PARAFAC (CP) decomposition to factorize each tensor into triple vectors that express local feature distributions along spatial axes and compactly encode a local neural field.

Tensor Decomposition

PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics

1 code implementation ICLR 2021 Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B. Tenenbaum, Chuang Gan

Experimental results suggest that 1) RL-based approaches struggle to solve most of the tasks efficiently; 2) gradient-based approaches, by optimizing open-loop control sequences with the built-in differentiable physics engine, can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning.

Reinforcement Learning (RL)

CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

1 code implementation ICCV 2021 Yijia Weng, He Wang, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas J. Guibas

For the first time, we propose a unified framework that can handle 9DoF pose tracking for novel rigid object instances as well as per-part pose tracking for articulated objects from known categories.

Pose Tracking

Normal Assisted Stereo Depth Estimation

1 code implementation CVPR 2020 Uday Kusupati, Shuo Cheng, Rui Chen, Hao Su

We couple the learning of a multi-view normal estimation module and a multi-view depth estimation module.

Stereo Depth Estimation

Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness

1 code implementation CVPR 2020 Shuo Cheng, Zexiang Xu, Shilin Zhu, Zhuwen Li, Li Erran Li, Ravi Ramamoorthi, Hao Su

In contrast, we propose adaptive thin volumes (ATVs); in an ATV, the depth hypothesis of each plane is spatially varying, which adapts to the uncertainties of previous per-pixel depth predictions.

3D Reconstruction Point Clouds

Deductive Verification of Chain-of-Thought Reasoning

1 code implementation NeurIPS 2023 Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, Hao Su

In light of this, we propose to decompose a reasoning verification process into a series of step-by-step subprocesses, each only receiving their necessary context and premises.

Logical Reasoning

Generic 3D Diffusion Adapter Using Controlled Multi-View Editing

1 code implementation18 Mar 2024 Hansheng Chen, Ruoxi Shi, Yulin Liu, Bokui Shen, Jiayuan Gu, Gordon Wetzstein, Hao Su, Leonidas Guibas

Open-domain 3D object synthesis has been lagging behind image synthesis due to limited data and higher computational complexity.

Image Generation Image to 3D +2

NeuTex: Neural Texture Mapping for Volumetric Neural Rendering

1 code implementation CVPR 2021 Fanbo Xiang, Zexiang Xu, Miloš Hašan, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Hao Su

We achieve this by introducing a 3D-to-2D texture mapping (or surface parameterization) network into volumetric representations.

Neural Rendering

DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates

1 code implementation CVPR 2021 Minghua Liu, Minhyuk Sung, Radomir Mech, Hao Su

Given a collection of 3D meshes of a category and their deformation handles (control points), our method learns a set of meta-handles for each shape, which are represented as combinations of the given handles.

MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations

1 code implementation12 Dec 2022 Nicklas Hansen, Yixin Lin, Hao Su, Xiaolong Wang, Vikash Kumar, Aravind Rajeswaran

We identify key ingredients for leveraging demonstrations in model learning -- policy pretraining, targeted exploration, and oversampling of demonstration data -- which forms the three phases of our model-based RL framework.

Model-based Reinforcement Learning reinforcement-learning +1

BiPointNet: Binary Neural Network for Point Clouds

1 code implementation ICLR 2021 Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Liu, Hao Su

To alleviate the resource constraint for real-time point cloud applications that run on edge devices, in this paper we present BiPointNet, the first model binarization approach for efficient deep learning on point clouds.

Binarization

MARVEL: Raster Manga Vectorization via Primitive-wise Deep Reinforcement Learning

1 code implementation10 Oct 2021 Hao Su, Jianwei Niu, Xuefeng Liu, Jiahe Cui, Ji Wan

Manga is a fashionable Japanese-style comic form that is composed of black-and-white strokes and is generally displayed as raster images on digital devices.

reinforcement-learning Reinforcement Learning (RL) +1

Learning Non-Lambertian Object Intrinsics across ShapeNet Categories

1 code implementation CVPR 2017 Jian Shi, Yue Dong, Hao Su, Stella X. Yu

Rendered with realistic environment maps, millions of synthetic images of objects and their corresponding albedo, shading, and specular ground-truth images are used to train an encoder-decoder CNN.

Object

Deep Part Induction from Articulated Object Pairs

1 code implementation19 Sep 2018 Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, Leonidas Guibas

In this paper, we explore how the observation of different articulation states provides evidence for part structure and motion of 3D objects.

Object

Chain-of-Thought Predictive Control

1 code implementation3 Apr 2023 Zhiwei Jia, Fangchen Liu, Vineet Thumuluri, Linghao Chen, Zhiao Huang, Hao Su

We study generalizable policy learning from demonstrations for complex low-level control tasks (e. g., contact-rich object manipulations).

Imitation Learning

Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions

1 code implementation NeurIPS 2018 Minhyuk Sung, Hao Su, Ronald Yu, Leonidas Guibas

Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes.

Segmentation

Distilling Large Vision-Language Model with Out-of-Distribution Generalizability

1 code implementation ICCV 2023 Xuanlin Li, Yunhao Fang, Minghua Liu, Zhan Ling, Zhuowen Tu, Hao Su

Model distillation, the process of creating smaller, faster models that maintain the performance of larger models, is a promising direction towards the solution.

Few-Shot Image Classification Knowledge Distillation +7

StructEdit: Learning Structural Shape Variations

1 code implementation CVPR 2020 Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas

Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation.

Adversarial shape perturbations on 3D point clouds

2 code implementations16 Aug 2019 Daniel Liu, Ronald Yu, Hao Su

The importance of training robust neural network grows as 3D data is increasingly utilized in deep learning for vision tasks in robotics, drone control, and autonomous driving.

3D Point Cloud Classification Autonomous Driving +1

S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenes

1 code implementation31 Oct 2019 Yuzhe Qin, Rui Chen, Hao Zhu, Meng Song, Jing Xu, Hao Su

Grasping is among the most fundamental and long-lasting problems in robotics study.

Compositionally Generalizable 3D Structure Prediction

1 code implementation4 Dec 2020 Songfang Han, Jiayuan Gu, Kaichun Mo, Li Yi, Siyu Hu, Xuejin Chen, Hao Su

However, there remains a much more difficult and under-explored issue on how to generalize the learned skills over unseen object categories that have very different shape geometry distributions.

3D Shape Reconstruction Object +1

O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning

1 code implementation29 Jun 2021 Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, Leonidas Guibas

Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.

Object

Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?

1 code implementation CVPR 2019 Shilin Zhu, Xin Dong, Hao Su

Binary neural networks (BNN) have been studied extensively since they run dramatically faster at lower memory and power consumption than floating-point networks, thanks to the efficiency of bit operations.

ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling

1 code implementation6 Aug 2017 Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas

The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input.

Graphics I.3.5

Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds

1 code implementation14 Oct 2022 Minghua Liu, Xuanlin Li, Zhan Ling, Yangyan Li, Hao Su

We study how choices of input point cloud coordinate frames impact learning of manipulation skills from 3D point clouds.

3D Point Cloud Reinforcement Learning Imitation Learning +2

Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers

1 code implementation10 Jan 2019 Daniel Liu, Ronald Yu, Hao Su

We present a preliminary evaluation of adversarial attacks on deep 3D point cloud classifiers, namely PointNet and PointNet++, by evaluating both white-box and black-box adversarial attacks that were proposed for 2D images and extending those attacks to reduce the perceptibility of the perturbations in 3D space.

3D Object Classification General Classification +1

DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfaces

2 code implementations12 Jan 2019 Jun Gao, Chengcheng Tang, Vignesh Ganapathi-Subramanian, Jiahui Huang, Hao Su, Leonidas J. Guibas

Reconstruction of geometry based on different input modes, such as images or point clouds, has been instrumental in the development of computer aided design and computer graphics.

Abstract-to-Executable Trajectory Translation for One-Shot Task Generalization

1 code implementation14 Oct 2022 Stone Tao, Xiaochen Li, Tongzhou Mu, Zhiao Huang, Yuzhe Qin, Hao Su

In the abstract environment, complex dynamics such as physical manipulation are removed, making abstract trajectories easier to generate.

Few-Shot Imitation Learning Reinforcement Learning (RL)

Particle Cloud Generation with Message Passing Generative Adversarial Networks

2 code implementations NeurIPS 2021 Raghav Kansal, Javier Duarte, Hao Su, Breno Orzari, Thiago Tomei, Maurizio Pierini, Mary Touranakou, Jean-Roch Vlimant, Dimitrios Gunopulos

We propose JetNet as a novel point-cloud-style dataset for the ML community to experiment with, and set MPGAN as a benchmark to improve upon for future generative models.

Adversarial Defense by Stratified Convolutional Sparse Coding

1 code implementation CVPR 2019 Bo Sun, Nian-hsuan Tsai, Fangchen Liu, Ronald Yu, Hao Su

We propose an adversarial defense method that achieves state-of-the-art performance among attack-agnostic adversarial defense methods while also maintaining robustness to input resolution, scale of adversarial perturbation, and scale of dataset size.

Adversarial Defense

Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous Graph Neural Networks

1 code implementation26 Oct 2020 Hao Tang, Zhiao Huang, Jiayuan Gu, Bao-liang Lu, Hao Su

Current graph neural networks (GNNs) lack generalizability with respect to scales (graph sizes, graph diameters, edge weights, etc..) when solving many graph analysis problems.

On the Efficacy of 3D Point Cloud Reinforcement Learning

1 code implementation11 Jun 2023 Zhan Ling, Yunchao Yao, Xuanlin Li, Hao Su

Recent studies on visual reinforcement learning (visual RL) have explored the use of 3D visual representations.

3D Point Cloud Reinforcement Learning Inductive Bias +3

Mapping State Space using Landmarks for Universal Goal Reaching

1 code implementation NeurIPS 2019 Zhiao Huang, Fangchen Liu, Hao Su

An agent that has well understood the environment should be able to apply its skills for any given goals, leading to the fundamental problem of learning the Universal Value Function Approximator (UVFA).

On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning

1 code implementation19 Oct 2022 Yifan Xu, Nicklas Hansen, ZiRui Wang, Yung-Chieh Chan, Hao Su, Zhuowen Tu

Reinforcement Learning (RL) algorithms can solve challenging control problems directly from image observations, but they often require millions of environment interactions to do so.

Atari Games 100k Model-based Reinforcement Learning +2

How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?

1 code implementation28 Mar 2019 Quan Vuong, Sharad Vikram, Hao Su, Sicun Gao, Henrik I. Christensen

A human-specified design choice in domain randomization is the form and parameters of the distribution of simulated environments.

Reinforcement Learning (RL)

Information-Theoretic Local Minima Characterization and Regularization

1 code implementation ICML 2020 Zhiwei Jia, Hao Su

Recent advances in deep learning theory have evoked the study of generalizability across different local minima of deep neural networks (DNNs).

Learning Theory

Improving Policy Optimization with Generalist-Specialist Learning

1 code implementation26 Jun 2022 Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao Su

Generalization in deep reinforcement learning over unseen environment variations usually requires policy learning over a large set of diverse training variations.

Imitation Learning

Beyond Holistic Object Recognition: Enriching Image Understanding with Part States

no code implementations CVPR 2018 Cewu Lu, Hao Su, Yongyi Lu, Li Yi, Chi-Keung Tang, Leonidas Guibas

Important high-level vision tasks such as human-object interaction, image captioning and robotic manipulation require rich semantic descriptions of objects at part level.

Human-Object Interaction Detection Image Captioning +1

SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

no code implementations CVPR 2017 Li Yi, Hao Su, Xingwen Guo, Leonidas Guibas

To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases.

3D Part Segmentation

Density Estimation via Discrepancy

no code implementations23 Sep 2015 Kun Yang, Hao Su, Wing Hung Wang

Given i. i. d samples from some unknown continuous density on hyper-rectangle $[0, 1]^d$, we attempt to learn a piecewise constant function that approximates this underlying density non-parametrically.

Density Estimation

Pose Estimation Based on 3D Models

no code implementations20 Jun 2015 Chuiwen Ma, Hao Su, Liang Shi

In this paper, we proposed a pose estimation system based on rendered image training set, which predicts the pose of objects in real image, with knowledge of object category and tight bounding box.

General Classification Multi-class Classification +1

3D-Assisted Image Feature Synthesis for Novel Views of an Object

no code implementations26 Nov 2014 Hao Su, Fan Wang, Li Yi, Leonidas Guibas

In this paper, given a single input image of an object, we synthesize new features for other views of the same object.

Image Retrieval Object +1

Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA

no code implementations4 Oct 2018 Cheng Fu, Shilin Zhu, Hao Su, Ching-En Lee, Jishen Zhao

Thus there does exist redundancy that can be exploited to further reduce the amount of on-chip computations.

ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving

no code implementations CVPR 2019 Xibin Song, Peng Wang, Dingfu Zhou, Rui Zhu, Chenye Guan, Yuchao Dai, Hao Su, Hongdong Li, Ruigang Yang

Specifically, we first segment each car with a pre-trained Mask R-CNN, and then regress towards its 3D pose and shape based on a deformable 3D car model with or without using semantic keypoints.

3D Car Instance Understanding Autonomous Driving

A Main/Subsidiary Network Framework for Simplifying Binary Neural Network

no code implementations11 Dec 2018 Yinghao Xu, Xin Dong, Yudian Li, Hao Su

To reduce memory footprint and run-time latency, techniques such as neural network pruning and binarization have been explored separately.

Binarization Image Classification +1

Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification

no code implementations NeurIPS 2010 Li-Jia Li, Hao Su, Li Fei-Fei, Eric P. Xing

Robust low-level image features have been proven to be effective representations for a variety of visual recognition tasks such as object recognition and scene classification; but pixels, or even local image patches, carry little semantic meanings.

General Classification Object +2

Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning

no code implementations CVPR 2018 Chuang Gan, Boqing Gong, Kun Liu, Hao Su, Leonidas J. Guibas

In addition, we also find that a progressive training strategy can foster a better neural network for the video recognition task than blindly pooling the distinct sources of geometry cues together.

Action Recognition Representation Learning +5

Multilinear Hyperplane Hashing

no code implementations CVPR 2016 Xianglong Liu, Xinjie Fan, Cheng Deng, Zhujin Li, Hao Su, DaCheng Tao

Despite its successful progress in classic point-to-point search, there are few studies regarding point-to-hyperplane search, which has strong practical capabilities of scaling up in many applications like active learning with SVMs.

Active Learning Quantization

3D-Assisted Feature Synthesis for Novel Views of an Object

no code implementations ICCV 2015 Hao Su, Fan Wang, Eric Yi, Leonidas J. Guibas

Comparing two images from different views has been a long-standing challenging problem in computer vision, as visual features are not stable under large view point changes.

Image Retrieval Object +1

Model Imitation for Model-Based Reinforcement Learning

no code implementations25 Sep 2019 Yueh-Hua Wu, Ting-Han Fan, Peter J. Ramadge, Hao Su

Based on the claim, we propose to learn the transition model by matching the distributions of multi-step rollouts sampled from the transition model and the real ones via WGAN.

Model-based Reinforcement Learning reinforcement-learning +1

Multi-view PointNet for 3D Scene Understanding

no code implementations30 Sep 2019 Maximilian Jaritz, Jiayuan Gu, Hao Su

Fusion of 2D images and 3D point clouds is important because information from dense images can enhance sparse point clouds.

3D Instance Segmentation 3D Semantic Segmentation +2

Regret Bounds for Discounted MDPs

no code implementations12 Feb 2020 Shuang Liu, Hao Su

We give motivations and derive lower and upper bounds for such measures.

Q-Learning Reinforcement Learning (RL)

MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing

no code implementations22 Apr 2020 Hao Su, Jianwei Niu, Xuefeng Liu, Qingfeng Li, Jiahe Cui, Ji Wan

Manga is a world popular comic form originated in Japan, which typically employs black-and-white stroke lines and geometric exaggeration to describe humans' appearances, poses, and actions.

Generative Adversarial Network Translation

Deep Photon Mapping

no code implementations25 Apr 2020 Shilin Zhu, Zexiang Xu, Henrik Wann Jensen, Hao Su, Ravi Ramamoorthi

This network is easy to incorporate in many previous photon mapping methods (by simply swapping the kernel density estimator) and can produce high-quality reconstructions of complex global illumination effects like caustics with an order of magnitude fewer photons compared to previous photon mapping methods.

Denoising Density Estimation

Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks

no code implementations12 Jun 2020 He Wang, Zetian Jiang, Li Yi, Kaichun Mo, Hao Su, Leonidas J. Guibas

We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics.

Clustering valid

Balance Scene Learning Mechanism for Offshore and Inshore Ship Detection in SAR Images

no code implementations21 Jul 2020 Tianwen Zhang, Xiaoling Zhang, Jun Shi, Shunjun Wei, Jianguo Wang, Jianwei Li, Hao Su, Yue Zhou

Huge imbalance of different scenes' sample numbers seriously reduces Synthetic Aperture Radar (SAR) ship detection accuracy.

Weakly-supervised 3D Shape Completion in the Wild

no code implementations ECCV 2020 Jiayuan Gu, Wei-Chiu Ma, Sivabalan Manivasagam, Wenyuan Zeng, ZiHao Wang, Yuwen Xiong, Hao Su, Raquel Urtasun

3D shape completion for real data is important but challenging, since partial point clouds acquired by real-world sensors are usually sparse, noisy and unaligned.

Point Cloud Registration Pose Estimation

Photon-Driven Neural Path Guiding

no code implementations5 Oct 2020 Shilin Zhu, Zexiang Xu, Tiancheng Sun, Alexandr Kuznetsov, Mark Meyer, Henrik Wann Jensen, Hao Su, Ravi Ramamoorthi

To fully make use of our deep neural network, we partition the scene space into an adaptive hierarchical grid, in which we apply our network to reconstruct high-quality sampling distributions for any local region in the scene.

Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous GNNs

no code implementations NeurIPS 2020 Hao Tang, Zhiao Huang, Jiayuan Gu, Bao-liang Lu, Hao Su

Current graph neural networks (GNNs) lack generalizability with respect to scales (graph sizes, graph diameters, edge weights, etc..) when solving many graph analysis problems.

An End-to-end Method for Producing Scanning-robust Stylized QR Codes

no code implementations16 Nov 2020 Hao Su, Jianwei Niu, Xuefeng Liu, Qingfeng Li, Ji Wan, Mingliang Xu, Tao Ren

Quick Response (QR) code is one of the most worldwide used two-dimensional codes.~Traditional QR codes appear as random collections of black-and-white modules that lack visual semantics and aesthetic elements, which inspires the recent works to beautify the appearances of QR codes.

Style Transfer

Medical Robots for Infectious Diseases: Lessons and Challenges from the COVID-19 Pandemic

no code implementations14 Dec 2020 Antonio Di Lallo, Robin R. Murphy, Axel Krieger, Junxi Zhu, Russell H. Taylor, Hao Su

Medical robots can play an important role in mitigating the spread of infectious diseases and delivering quality care to patients during the COVID-19 pandemic.

Robotics

One-shot Learning with Absolute Generalization

1 code implementation28 May 2021 Hao Su

One-shot learning is proposed to make a pretrained classifier workable on a new dataset based on one labeled samples from each pattern.

One-Shot Learning

Single RGB-D Camera Teleoperation for General Robotic Manipulation

no code implementations28 Jun 2021 Quan Vuong, Yuzhe Qin, Runlin Guo, Xiaolong Wang, Hao Su, Henrik Christensen

We propose a teleoperation system that uses a single RGB-D camera as the human motion capture device.

Transfer Value or Policy? A Value-centric Framework Towards Transferrable Continuous Reinforcement Learning

no code implementations27 Sep 2018 Xingchao Liu, Tongzhou Mu, Hao Su

In this paper, we investigate the problem of transfer learning across environments with different dynamics while accomplishing the same task in the continuous control domain.

Continuous Control Reinforcement Learning (RL) +1

Pre-training as Batch Meta Reinforcement Learning with tiMe

no code implementations25 Sep 2019 Quan Vuong, Shuang Liu, Minghua Liu, Kamil Ciosek, Hao Su, Henrik Iskov Christensen

Combining ideas from Batch RL and Meta RL, we propose tiMe, which learns distillation of multiple value functions and MDP embeddings from only existing data.

Meta Reinforcement Learning reinforcement-learning +1

ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation

no code implementations CVPR 2022 Isabella Liu, Edward Yang, Jianyu Tao, Rui Chen, Xiaoshuai Zhang, Qing Ran, Zhu Liu, Hao Su

First, we demonstrate the transferability of our method to out-of-distribution real data by using a mixed domain learning strategy.

Provably Efficient Kernelized Q-Learning

no code implementations21 Apr 2022 Shuang Liu, Hao Su

We propose and analyze a kernelized version of Q-learning.

Q-Learning

From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation

no code implementations26 Apr 2022 Yuzhe Qin, Hao Su, Xiaolong Wang

We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand.

Imitation Learning

Contact Points Discovery for Soft-Body Manipulations with Differentiable Physics

no code implementations ICLR 2022 Sizhe Li, Zhiao Huang, Tao Du, Hao Su, Joshua B. Tenenbaum, Chuang Gan

Extensive experimental results suggest that: 1) on multi-stage tasks that are infeasible for the vanilla differentiable physics solver, our approach discovers contact points that efficiently guide the solver to completion; 2) on tasks where the vanilla solver performs sub-optimally or near-optimally, our contact point discovery method performs better than or on par with the manipulation performance obtained with handcrafted contact points.

Multi-skill Mobile Manipulation for Object Rearrangement

no code implementations6 Sep 2022 Jiayuan Gu, Devendra Singh Chaplot, Hao Su, Jitendra Malik

To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks.

Object

LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds

no code implementations14 Oct 2022 Minghua Liu, Yin Zhou, Charles R. Qi, Boqing Gong, Hao Su, Dragomir Anguelov

Our method co-designs an efficient labeling process with semi/weakly supervised learning and is applicable to nearly any 3D semantic segmentation backbones.

3D Semantic Segmentation Autonomous Driving +3

MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field

no code implementations10 Mar 2023 Kaizhi Yang, Xiaoshuai Zhang, Zhiao Huang, Xuejin Chen, Zexiang Xu, Hao Su

Under the Lagrangian view, we parameterize the scene motion by tracking the trajectory of particles on objects.

3D scene Editing

Boosting Reinforcement Learning and Planning with Demonstrations: A Survey

no code implementations23 Mar 2023 Tongzhou Mu, Hao Su

Although reinforcement learning has seen tremendous success recently, this kind of trial-and-error learning can be impractical or inefficient in complex environments.

Decision Making reinforcement-learning

DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics

no code implementations27 Mar 2023 Sizhe Li, Zhiao Huang, Tao Chen, Tao Du, Hao Su, Joshua B. Tenenbaum, Chuang Gan

Reinforcement learning approaches for dexterous rigid object manipulation would struggle in this setting due to the complexity of physics interaction with deformable objects.

Deformable Object Manipulation Object

EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration

no code implementations2 May 2023 Linghao Chen, Yuzhe Qin, Xiaowei Zhou, Hao Su

Hand-eye calibration is a critical task in robotics, as it directly affects the efficacy of critical operations such as manipulation and grasping.

AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System

no code implementations10 Jul 2023 Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, Dieter Fox

For real-world experiments, AnyTeleop can outperform a previous system that was designed for a specific robot hardware with a higher success rate, using the same robot.

Imitation Learning

3Deformer: A Common Framework for Image-Guided Mesh Deformation

no code implementations19 Jul 2023 Hao Su, Xuefeng Liu, Jianwei Niu, Ji Wan, Xinghao Wu

Unlike these studies, our 3Deformer is a non-training and common framework, which only requires supervision of readily-available semantic images, and is compatible with editing various objects unlimited by datasets.

Reparameterized Policy Learning for Multimodal Trajectory Optimization

no code implementations20 Jul 2023 Zhiao Huang, Litian Liang, Zhan Ling, Xuanlin Li, Chuang Gan, Hao Su

We then present a practical model-based RL method, called Reparameterized Policy Gradient (RPG), which leverages the multimodal policy parameterization and learned world model to achieve strong exploration capabilities and high data efficiency.

Reinforcement Learning (RL)

OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objects

no code implementations NeurIPS 2023 Isabella Liu, Linghao Chen, Ziyang Fu, Liwen Wu, Haian Jin, Zhong Li, Chin Ming Ryan Wong, Yi Xu, Ravi Ramamoorthi, Zexiang Xu, Hao Su

We introduce OpenIllumination, a real-world dataset containing over 108K images of 64 objects with diverse materials, captured under 72 camera views and a large number of different illuminations.

Foreground Segmentation Inverse Rendering

FG-NeRF: Flow-GAN based Probabilistic Neural Radiance Field for Independence-Assumption-Free Uncertainty Estimation

no code implementations28 Sep 2023 Songlin Wei, Jiazhao Zhang, Yang Wang, Fanbo Xiang, Hao Su, He Wang

Existing works rely on the independence assumption of points in the radiance field or the pixels in input views to obtain tractable forms of the probability density function.

One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion

no code implementations14 Nov 2023 Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, Hao Su

Recent advancements in open-world 3D object generation have been remarkable, with image-to-3D methods offering superior fine-grained control over their text-to-3D counterparts.

Image Generation Image to 3D +1

DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics

no code implementations NeurIPS 2023 Zhiao Huang, Feng Chen, Yewen Pu, Chunru Lin, Hao Su, Chuang Gan

Combining gradient-based trajectory optimization with differentiable physics simulation is an efficient technique for solving soft-body manipulation problems.

valid

Robo360: A 3D Omnispective Multi-Material Robotic Manipulation Dataset

no code implementations9 Dec 2023 Litian Liang, Liuyu Bian, Caiwei Xiao, Jialin Zhang, Linghao Chen, Isabella Liu, Fanbo Xiang, Zhiao Huang, Hao Su

Building robots that can automate labor-intensive tasks has long been the core motivation behind the advancements in computer vision and the robotics community.

Representation Learning

ZeroRF: Fast Sparse View 360° Reconstruction with Zero Pretraining

no code implementations14 Dec 2023 Ruoxi Shi, Xinyue Wei, Cheng Wang, Hao Su

We present ZeroRF, a novel per-scene optimization method addressing the challenge of sparse view 360{\deg} reconstruction in neural field representations.

Image Generation

Constrained Online Two-stage Stochastic Optimization: Algorithm with (and without) Predictions

no code implementations2 Jan 2024 Piao Hu, Jiashuo Jiang, Guodong Lyu, Hao Su

When the model parameters are drawn from unknown non-stationary distributions and we are given machine-learned predictions of the distributions, we develop a new algorithm from our framework with a regret $O(W_T+\sqrt{T})$, where $W_T$ measures the total inaccuracy of the machine-learned predictions.

Stochastic Optimization

CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation

no code implementations22 Feb 2024 Jun Wang, Yuzhe Qin, Kaiming Kuang, Yigit Korkmaz, Akhilan Gurumoorthy, Hao Su, Xiaolong Wang

We introduce CyberDemo, a novel approach to robotic imitation learning that leverages simulated human demonstrations for real-world tasks.

Data Augmentation Imitation Learning

Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D

no code implementations27 Mar 2024 Mukund Varma T, Peihao Wang, Zhiwen Fan, Zhangyang Wang, Hao Su, Ravi Ramamoorthi

In recent years, there has been an explosion of 2D vision models for numerous tasks such as semantic segmentation, style transfer or scene editing, enabled by large-scale 2D image datasets.

Cannot find the paper you are looking for? You can Submit a new open access paper.