Search Results for author: He Wang

Found 140 papers, 63 papers with code

Saliency Detection via Cellular Automata

no code implementations CVPR 2015 Yao Qin, Huchuan Lu, Yiqun Xu, He Wang

In this paper, we introduce Cellular Automata--a dynamic evolution model to intuitively detect the salient object.

Saliency Detection

“Congruent” and “Opposite” Neurons: Sisters for Multisensory Integration and Segregation

no code implementations NeurIPS 2016 Wen-Hao Zhang, He Wang, K. Y. Michael Wong, Si Wu

Mimicking the experimental protocol, our model reproduces the characteristics of congruent and opposite neurons, and demonstrates that in each module, the sisters of congruent and opposite neurons can jointly achieve optimal multisensory information integration and segregation.

The Online Saddle Point Problem and Online Convex Optimization with Knapsacks

no code implementations21 Jun 2018 Adrian Rivera, He Wang, Huan Xu

We relate this problem to the online saddle point problem and establish $O(\sqrt{T})$ regret using a primal-dual algorithm.

Novelty Detection Meets Collider Physics

no code implementations26 Jul 2018 Jan Hajer, Ying-Ying Li, Tao Liu, He Wang

Complementary to supervised learning, it allows to analyze data model-independently.

Clustering Novelty Detection

Globally Continuous and Non-Markovian Activity Analysis from Videos

no code implementations11 Oct 2018 He Wang, Carol O'Sullivan

By combining these patterns with detailed environment information, we interpret the semantics of activities and report anomalies.

Learning Generalizable Physical Dynamics of 3D Rigid Objects

no code implementations2 Jan 2019 Davis Rempe, Srinath Sridhar, He Wang, Leonidas J. Guibas

In this work, we focus on predicting the dynamics of 3D rigid objects, in particular an object's final resting position and total rotation when subjected to an impulsive force.

Autonomous Vehicles Object +1

DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark

no code implementations23 Apr 2019 Jianwu Fang, Dingxin Yan, Jiahuan Qiao, Jianru Xue, He Wang, Sen Li

Driver attention prediction is currently becoming the focus in safe driving research community, such as the DR(eye)VE project and newly emerged Berkeley DeepDrive Attention (BDD-A) database in critical situations.

Driver Attention Monitoring

Large Scale Markov Decision Processes with Changing Rewards

no code implementations NeurIPS 2019 Adrian Rivera Cardoso, He Wang, Huan Xu

We consider Markov Decision Processes (MDPs) where the rewards are unknown and may change in an adversarial manner.

Competing Against Equilibria in Zero-Sum Games with Evolving Payoffs

1 code implementation17 Jul 2019 Adrian Rivera Cardoso, Jacob Abernethy, He Wang, Huan Xu

Finding the Nash Equilibrium (NE) of a two player zero-sum game is core to many problems in statistics, optimization, and economics, and for a fixed game matrix this can be easily reduced to solving a linear program.

Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling

no code implementations20 Aug 2019 He Wang, Edmond S. L. Ho, Hubert P. H. Shum, Zhanxing Zhu

In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications.

Denoising Time Series Analysis

Gravitational wave signal recognition of O1 data by deep learning

1 code implementation30 Sep 2019 He Wang, Zhoujian Cao, Xiaolin Liu, Shichao Wu, Jian-Yang Zhu

Our adjusted convolutional neural network admits comparable accuracy and efficiency of signal recognition as other deep learning works published in the literature.

Instrumentation and Methods for Astrophysics General Relativity and Quantum Cosmology

Bimodular continuous attractor neural networks with static and moving stimuli

no code implementations16 Oct 2019 Min Yan, Wen-Hao Zhang, He Wang, K. Y. Michael Wong

We found that when bumps coexist in both modules, the position of each bump is shifted towards the other input when the intermodular couplings are excitatory and is shifted away when inhibitory.

Causal Inference

SMART: Skeletal Motion Action Recognition aTtack

no code implementations16 Nov 2019 He Wang, Feixiang He, Zhexi Peng, Yong-Liang Yang, Tianjia Shao, Kun Zhou, David Hogg

In this paper, we propose a method, SMART, to attack action recognizers which rely on 3D skeletal motions.

Action Recognition Adversarial Attack +2

Category-Level Articulated Object Pose Estimation

2 code implementations CVPR 2020 Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song

We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space.

Object Pose Estimation

Predicting the Physical Dynamics of Unseen 3D Objects

1 code implementation16 Jan 2020 Davis Rempe, Srinath Sridhar, He Wang, Leonidas J. Guibas

Experiments show that we can accurately predict the changes in state for unseen object geometries and initial conditions.

Object

Human-like Planning for Reaching in Cluttered Environments

1 code implementation28 Feb 2020 Mohamed Hasan, Matthew Warburton, Wisdom C. Agboh, Mehmet R. Dogar, Matteo Leonetti, He Wang, Faisal Mushtaq, Mark Mon-Williams, Anthony G. Cohn

From this, we devised a qualitative representation of the task space to abstract the decision making, irrespective of the number of obstacles.

Decision Making

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

1 code implementation ECCV 2020 Kaichun Mo, He Wang, Xinchen Yan, Leonidas J. Guibas

3D generative shape modeling is a fundamental research area in computer vision and interactive computer graphics, with many real-world applications.

3D Shape Generation

Curriculum DeepSDF

1 code implementation ECCV 2020 Yueqi Duan, Haidong Zhu, He Wang, Li Yi, Ram Nevatia, Leonidas J. Guibas

When learning to sketch, beginners start with simple and flexible shapes, and then gradually strive for more complex and accurate ones in the subsequent training sessions.

3D Shape Representation Representation Learning

SAPIEN: A SimulAted Part-based Interactive ENvironment

1 code implementation CVPR 2020 Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su

To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.

Attribute

MeshingNet: A New Mesh Generation Method based on Deep Learning

no code implementations15 Apr 2020 Zheyan Zhang, Yongxing Wang, Peter K. Jimack, He Wang

The framework that we have developed is based around training an artificial neural network (ANN) to guide standard mesh generation software, based upon a prediction of the required local mesh density throughout the domain.

Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance

no code implementations29 Apr 2020 Feixiang He, Yuanhang Xiang, Xi Zhao, He Wang

The method takes as input raw and noisy data with highly mixed multi-dimensional (space, time and dynamics) information, and automatically structure it by learning the correlations among these dimensions.

Data Visualization

Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks

no code implementations12 Jun 2020 He Wang, Zetian Jiang, Li Yi, Kaichun Mo, Hao Su, Leonidas J. Guibas

We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics.

Clustering valid

Object-Centric Multi-View Aggregation

no code implementations20 Jul 2020 Shubham Tulsiani, Or Litany, Charles R. Qi, He Wang, Leonidas J. Guibas

We present an approach for aggregating a sparse set of views of an object in order to compute a semi-implicit 3D representation in the form of a volumetric feature grid.

Novel View Synthesis Object +1

Dynamic Future Net: Diversified Human Motion Generation

no code implementations25 Aug 2020 Wenheng Chen, He Wang, Yi Yuan, Tianjia Shao, Kun Zhou

We evaluate our model on a wide range of motions and compare it with the state-of-the-art methods.

Constant Regret Re-solving Heuristics for Price-based Revenue Management

no code implementations7 Sep 2020 Yining Wang, He Wang

First, we prove that a natural re-solving heuristic attains $O(1)$ regret compared to the value of the optimal policy.

Management

IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration

2 code implementations11 Oct 2020 Ziyi Wu, Yueqi Duan, He Wang, Qingnan Fan, Leonidas J. Guibas

The former aims to recover the surface of point cloud through implicit function, while the latter encourages evenly-distributed points.

3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection

2 code implementations CVPR 2021 He Wang, Yezhen Cong, Or Litany, Yue Gao, Leonidas J. Guibas

On KITTI, we are the first to demonstrate semi-supervised 3D object detection and our method surpasses a fully supervised baseline from 1. 8% to 7. 6% under different label ratios and categories.

3D Object Detection Autonomous Driving +1

Detection of magnetic gap in the topological surface states of MnBi2Te4

no code implementations31 Dec 2020 Haoran Ji, Yanzhao Liu, He Wang, Jiawei Luo, Jiaheng Li, Hao Li, Yang Wu, Yong Xu, Jian Wang

An essential ingredient to realize these quantum states is the magnetic gap in the topological surface states induced by the out-of-plane ferromagnetism on the surface of MnBi2Te4.

Materials Science

Single Image 3D Shape Retrieval via Cross-Modal Instance and Category Contrastive Learning

no code implementations ICCV 2021 Ming-Xian Lin, Jie Yang, He Wang, Yu-Kun Lai, Rongfei Jia, Binqiang Zhao, Lin Gao

Inspired by the great success in recent contrastive learning works on self-supervised representation learning, we propose a novel IBSR pipeline leveraging contrastive learning.

3D Shape Retrieval Contrastive Learning +4

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

1 code implementation CVPR 2021 Jiahui Huang, He Wang, Tolga Birdal, Minhyuk Sung, Federica Arrigoni, Shi-Min Hu, Leonidas Guibas

We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds.

Motion Estimation Motion Segmentation +1

Systematic electrochemical etching of various metal tips for tunneling spectroscopy and scanning probe microscopy

no code implementations18 Jan 2021 Jiawei Zhang, Pinyuan Wang, Xuao Zhang, Haoran Ji, Jiawei Luo, He Wang, Jian Wang

To ensure the reproducibility of experimental results, the fabrication of tips should be standardized, and a controllable and convenient system should be set up.

Materials Science

MoDL-QSM: Model-based Deep Learning for Quantitative Susceptibility Mapping

1 code implementation21 Jan 2021 Ruimin Feng, Jiayi Zhao, He Wang, Baofeng Yang, Jie Feng, Yuting Shi, Ming Zhang, Chunlei Liu, Yuyao Zhang, Jie Zhuang, Hongjiang Wei

However, there exists a mismatch between the observed phase and the theoretical forward phase estimated by the susceptibility label.

SSIM

In-game Residential Home Planning via Visual Context-aware Global Relation Learning

no code implementations8 Feb 2021 Lijuan Liu, Yin Yang, Yi Yuan, Tianjia Shao, He Wang, Kun Zhou

In this paper, we propose an effective global relation learning algorithm to recommend an appropriate location of a building unit for in-game customization of residential home complex.

Graph Generation Relation

High-order Differentiable Autoencoder for Nonlinear Model Reduction

no code implementations19 Feb 2021 Siyuan Shen, Yang Yin, Tianjia Shao, He Wang, Chenfanfu Jiang, Lei Lan, Kun Zhou

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation.

Vocal Bursts Intensity Prediction

Distributed Optimization with Coupling Constraints

no code implementations25 Feb 2021 Xuyang Wu, He Wang, Jie Lu

In this paper, we develop a novel distributed algorithm for addressing convex optimization with both nonlinear inequality and linear equality constraints, where the objective function can be a general nonsmooth convex function and all the constraints can be fully coupled.

Distributed Optimization Optimization and Control

Enhanced 3D Human Pose Estimation from Videos by using Attention-Based Neural Network with Dilated Convolutions

1 code implementation4 Mar 2021 Ruixu Liu, Ju Shen, He Wang, Chen Chen, Sen-ching Cheung, Vijayan K. Asari

In this work, we show a systematic design (from 2D to 3D) for how conventional networks and other forms of constraints can be incorporated into the attention framework for learning long-range dependencies for the task of pose estimation.

2D Pose Estimation 3D Human Pose Estimation

Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack

1 code implementation CVPR 2021 He Wang, Feixiang He, Zhexi Peng, Tianjia Shao, Yong-Liang Yang, Kun Zhou, David Hogg

In this paper, we examine the robustness of state-of-the-art action recognizers against adversarial attack, which has been rarely investigated so far.

Action Recognition Adversarial Attack +4

BASAR:Black-box Attack on Skeletal Action Recognition

1 code implementation CVPR 2021 Yunfeng Diao, Tianjia Shao, Yong-Liang Yang, Kun Zhou, He Wang

The robustness of skeleton-based activity recognizers has been questioned recently, which shows that they are vulnerable to adversarial attacks when the full-knowledge of the recognizer is accessible to the attacker.

Action Recognition Adversarial Attack +1

Data-Driven Optimization for Atlanta Police Zone Design

no code implementations30 Mar 2021 Shixiang Zhu, He Wang, Yao Xie

By analyzing data before and after the zone redesign, we show that the new design has reduced the response time to high priority 911 calls by 5. 8\% and the imbalance of police workload among different zones by 43\%.

Decentralized Statistical Inference with Unrolled Graph Neural Networks

1 code implementation4 Apr 2021 He Wang, Yifei Shen, Ziyuan Wang, Dongsheng Li, Jun Zhang, Khaled B. Letaief, Jie Lu

In this paper, we investigate the decentralized statistical inference problem, where a network of agents cooperatively recover a (structured) vector from private noisy samples without centralized coordination.

CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

1 code implementation ICCV 2021 Yijia Weng, He Wang, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas J. Guibas

For the first time, we propose a unified framework that can handle 9DoF pose tracking for novel rigid object instances as well as per-part pose tracking for articulated objects from known categories.

Pose Tracking

Maneuver-based Anchor Trajectory Hypotheses at Roundabouts

1 code implementation22 Apr 2021 Mohamed Hasan, Evangelos Paschalidis, Albert Solernou, He Wang, Gustav Markkula, Richard Romano

Accordingly, our model employs a set of maneuver-specific anchor trajectories that cover the space of possible outcomes at the roundabout.

motion prediction Navigate

Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds

no code implementations NeurIPS 2021 Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang

To reduce the huge amount of pose annotations needed for category-level learning, we propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.

Object Pose Estimation +1

Survey of Image Based Graph Neural Networks

no code implementations11 Jun 2021 Usman Nazir, He Wang, Murtaza Taj

In this survey paper, we analyze image based graph neural networks and propose a three-step classification approach.

Classification Superpixels

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

1 code implementation29 Jul 2021 Yanchao Yang, Hanxiang Ren, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas

Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.

Hallucination Inductive Bias +2

Unsupervised Image Generation with Infinite Generative Adversarial Networks

1 code implementation ICCV 2021 Hui Ying, He Wang, Tianjia Shao, Yin Yang, Kun Zhou

Image generation has been heavily investigated in computer vision, where one core research challenge is to generate images from arbitrarily complex distributions with little supervision.

Image Generation

Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation

no code implementations NeurIPS 2021 Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang

Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.

Object Pose Estimation +1

Domain Adaptation on Point Clouds via Geometry-Aware Implicits

1 code implementation CVPR 2022 Yuefan Shen, Yanchao Yang, Mi Yan, He Wang, Youyi Zheng, Leonidas Guibas

Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.

Autonomous Driving Unsupervised Domain Adaptation

Dynamics-aware Adversarial Attack of 3D Sparse Convolution Network

1 code implementation17 Dec 2021 An Tao, Yueqi Duan, He Wang, Ziyi Wu, Pengliang Ji, Haowen Sun, Jie zhou, Jiwen Lu

It results in a serious issue of lagged gradient, making the learned attack at the current step ineffective due to the architecture changes afterward.

3D Classification 3D Semantic Segmentation +2

ADeLA: Automatic Dense Labeling With Attention for Viewpoint Shift in Semantic Segmentation

no code implementations CVPR 2022 Hanxiang Ren, Yanchao Yang, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas J. Guibas

We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.

Hallucination Semantic Segmentation +1

Fine-grained differentiable physics: a yarn-level model for fabrics

1 code implementation ICLR 2022 Deshan Gong, Zhanxing Zhu, Andrew J. Bulpitt, He Wang

To this end, we propose several differentiable forces, whose counterparts in empirical physics are indifferentiable, to facilitate gradient-based learning.

Pose Guided Image Generation from Misaligned Sources via Residual Flow Based Correction

no code implementations2 Feb 2022 Jiawei Lu, He Wang, Tianjia Shao, Yin Yang, Kun Zhou

However, as source images are often misaligned due to the large disparities among the camera settings, strong assumptions have been made in the past with respect to the camera(s) or/and the object in interest, limiting the application of such techniques.

Pose-Guided Image Generation

PartAfford: Part-level Affordance Discovery from 3D Objects

no code implementations28 Feb 2022 Chao Xu, Yixin Chen, He Wang, Song-Chun Zhu, Yixin Zhu, Siyuan Huang

We propose a novel learning framework for PartAfford, which discovers part-level representations by leveraging only the affordance set supervision and geometric primitive regularization, without dense supervision.

Object

HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction

1 code implementation CVPR 2022 Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, Li Yi

We present HOI4D, a large-scale 4D egocentric dataset with rich annotations, to catalyze the research of category-level human-object interaction.

Action Segmentation Benchmarking +6

Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations

2 code implementations4 Mar 2022 Hao Shen, Weikang Wan, He Wang

Generalizable object manipulation skills are critical for intelligent and multi-functional robots to work in real-world complex scenes.

Imitation Learning

Defending Black-box Skeleton-based Human Activity Classifiers

2 code implementations9 Mar 2022 He Wang, Yunfeng Diao, Zichang Tan, Guodong Guo

Our method is featured by full Bayesian treatments of the clean data, the adversaries and the classifier, leading to (1) a new Bayesian Energy-based formulation of robust discriminative classifiers, (2) a new adversary sampling scheme based on natural motion manifolds, and (3) a new post-train Bayesian strategy for black-box defense.

Human Activity Recognition Time Series Analysis

CodedVTR: Codebook-based Sparse Voxel Transformer with Geometric Guidance

no code implementations CVPR 2022 Tianchen Zhao, Niansong Zhang, Xuefei Ning, He Wang, Li Yi, Yu Wang

We propose CodedVTR (Codebook-based Voxel TRansformer), which improves data efficiency and generalization ability for 3D sparse voxel transformers.

3D Semantic Segmentation

iPLAN: Interactive and Procedural Layout Planning

1 code implementation CVPR 2022 Feixiang He, Yanlong Huang, He Wang

However, the capability of involving humans into the loop has been largely ignored in existing methods which are mostly end-to-end approaches.

Image Generation Layout Design

FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering

no code implementations CVPR 2022 Yingda Yin, Yingcheng Cai, He Wang, Baoquan Chen

Inspired by the popular semi-supervised approach, FixMatch, we propose to leverage pseudo label filtering to facilitate the information flow from labeled data to unlabeled data in a teacher-student mutual learning framework.

Pseudo Label Pseudo Label Filtering +1

Multi-Robot Active Mapping via Neural Bipartite Graph Matching

no code implementations CVPR 2022 Kai Ye, Siyan Dong, Qingnan Fan, He Wang, Li Yi, Fei Xia, Jue Wang, Baoquan Chen

Previous approaches either choose the frontier as the goal position via a myopic solution that hinders the time efficiency, or maximize the long-term value via reinforcement learning to directly regress the goal position, but does not guarantee the complete map construction.

Graph Matching Position +2

Attention U-Net as a surrogate model for groundwater prediction

no code implementations9 Apr 2022 Maria Luisa Taccari, Jonathan Nuttall, Xiaohui Chen, He Wang, Bennie Minnema, Peter K. Jimack

This manuscript presents an Attention U-Net model that attempts to capture the fundamental input-output relations of the groundwater system and generates solutions of hydraulic head in the whole domain given a set of physical parameters and boundary conditions.

Real-time Controllable Motion Transition for Characters

no code implementations5 May 2022 Xiangjun Tang, He Wang, Bo Hu, Xu Gong, Ruifan Yi, Qilong Kou, Xiaogang Jin

Then, during generation, we design a transition model which is essentially a sampling strategy to sample from the learned manifold, based on the target frame and the aimed transition duration.

8D Parameters Estimation for Bistatic EMVS-MIMO Radar via the nested PARAFAC

no code implementations4 Jun 2022 Qianpeng Xie, He Wang, Yihang Du, Xiaoyi Pan, Feng Zhao

Firstly, the outer part PARAFAC algorithm was carried out to estimate the receive spatial response matrix and its first way factor matrix.

Underdetermined 2D-DOD and 2D-DOA Estimation for Bistatic Coprime EMVS-MIMO Radar: From the Difference Coarray Perspective

no code implementations6 Jun 2022 Qianpeng Xie, Yihang Du, He Wang, Xiaoyi Pan, Feng Zhao

Firstly, a 5-D tensor model was constructed by using the multi-dimensional space-time characteristics of the received data.

Space-based gravitational wave signal detection and extraction with deep neural network

1 code implementation15 Jul 2022 Tianyu Zhao, Ruoxi Lyu, He Wang, Zhoujian Cao, Zhixiang Ren

Space-based gravitational wave (GW) detectors will be able to observe signals from sources that are otherwise nearly impossible from current ground-based detection.

Human Trajectory Prediction via Neural Social Physics

1 code implementation21 Jul 2022 Jiangbei Yue, Dinesh Manocha, He Wang

Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters.

Inductive Bias Trajectory Prediction

Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects

1 code implementation7 Aug 2022 Qiyu Dai, Jiyao Zhang, Qiwei Li, Tianhao Wu, Hao Dong, Ziyuan Liu, Ping Tan, He Wang

Commercial depth sensors usually generate noisy and missing depths, especially on specular and transparent objects, which poses critical issues to downstream depth or point cloud-based tasks.

Pose Estimation Transparent objects

Talking Head from Speech Audio using a Pre-trained Image Generator

no code implementations9 Sep 2022 Mohammed M. Alghamdi, He Wang, Andrew J. Bulpitt, David C. Hogg

We train a recurrent neural network to map from speech utterances to displacements in the latent space of the image generator.

SSIM

Shape Completion with Points in the Shadow

1 code implementation17 Sep 2022 BoWen Zhang, Xi Zhao, He Wang, Ruizhen Hu

The core challenge is to generate plausible geometries to fill the unobserved part of the object based on a partial scan, which is under-constrained and suffers from a huge solution space.

Object Point Cloud Completion

Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild

no code implementations24 Sep 2022 Jiayi Chen, Mi Yan, Jiazhao Zhang, Yinzhen Xu, Xiaolong Li, Yijia Weng, Li Yi, Shuran Song, He Wang

We for the first time propose a point cloud based hand joint tracking network, HandTrackNet, to estimate the inter-frame hand joint motion.

hand-object pose Object +2

DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation

no code implementations6 Oct 2022 Ruicheng Wang, Jialiang Zhang, Jiayi Chen, Yinzhen Xu, Puhao Li, Tengyu Liu, He Wang

Robotic dexterous grasping is the first step to enable human-like dexterous object manipulation and thus a crucial robotic technology.

Object

Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing

1 code implementation8 Oct 2022 Yun Liu, Xiaomeng Xu, Weihang Chen, Haocheng Yuan, He Wang, Jing Xu, Rui Chen, Li Yi

When manipulating an object to accomplish complex tasks, humans rely on both vision and touch to keep track of the object's 6D pose.

hand-object pose Object +1

GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF

1 code implementation12 Oct 2022 Qiyu Dai, Yan Zhu, Yiran Geng, Ciyu Ruan, Jiazhao Zhang, He Wang

In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry.

LeNo: Adversarial Robust Salient Object Detection Networks with Learnable Noise

1 code implementation27 Oct 2022 He Wang, Lin Wan, He Tang

In general, LeNo consists of a simple shallow noise and noise estimation that embedded in the encoder and decoder of arbitrary SOD networks respectively.

Noise Estimation object-detection +3

Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack

4 code implementations21 Nov 2022 Yunfeng Diao, He Wang, Tianjia Shao, Yong-Liang Yang, Kun Zhou, David Hogg

Via BASAR, we find on-manifold adversarial samples are extremely deceitful and rather common in skeletal motions, in contrast to the common belief that adversarial samples only exist off-manifold.

Adversarial Attack Human Activity Recognition +2

3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification

no code implementations CVPR 2023 Jiazhao Zhang, Liu Dai, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, He Wang

However, leveraging 3D scene representation can be prohibitively unpractical for policy learning in this floor-level task, due to low sample efficiency and expensive computational cost.

Full-Body Articulated Human-Object Interaction

1 code implementation ICCV 2023 Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, Siyuan Huang

By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation to tackle the estimation of articulated object poses and shapes during whole-body interactions.

Action Recognition Human-Object Interaction Detection +3

Adaptive Zone-Aware Hierarchical Planner for Vision-Language Navigation

1 code implementation CVPR 2023 Chen Gao, Xingyu Peng, Mi Yan, He Wang, Lirong Yang, Haibing Ren, Hongsheng Li, Si Liu

In this paper, we propose an Adaptive Zone-aware Hierarchical Planner (AZHP) to explicitly divides the navigation process into two heterogeneous phases, i. e., sub-goal setting via zone partition/selection (high-level action) and sub-goal executing (low-level action), for hierarchical planning.

Vision-Language Navigation

Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance

1 code implementation28 Feb 2023 Xueyi Liu, Ji Zhang, Ruizhen Hu, Haibin Huang, He Wang, Li Yi

Category-level articulated object pose estimation aims to estimate a hierarchy of articulation-aware object poses of an unseen articulated object from a known category.

Disentanglement Object +1

UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy

no code implementations CVPR 2023 Yinzhen Xu, Weikang Wan, Jialiang Zhang, Haoran Liu, Zikang Shan, Hao Shen, Ruicheng Wang, Haoran Geng, Yijia Weng, Jiayi Chen, Tengyu Liu, Li Yi, He Wang

Trained on our synthesized large-scale dexterous grasp dataset, this model enables us to sample diverse and high-quality dexterous grasp poses for the object point cloud. For the second stage, we propose to replace the motion planning used in parallel gripper grasping with a goal-conditioned grasp policy, due to the complexity involved in dexterous grasping execution.

Motion Planning

A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimation

no code implementations3 Mar 2023 Yingda Yin, Yang Wang, He Wang, Baoquan Chen

Rotation Laplace distribution is robust to the disturbance of outliers and enforces much gradient to the low-error region, resulting in a better convergence.

regression

Category-Level Multi-Part Multi-Joint 3D Shape Assembly

no code implementations10 Mar 2023 Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas

A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.

Graph Learning Graph Representation Learning

The NPU-ASLP System for Audio-Visual Speech Recognition in MISP 2022 Challenge

no code implementations11 Mar 2023 Pengcheng Guo, He Wang, Bingshen Mu, Ao Zhang, Peikun Chen

This paper describes our NPU-ASLP system for the Audio-Visual Diarization and Recognition (AVDR) task in the Multi-modal Information based Speech Processing (MISP) 2022 Challenge.

Audio-Visual Speech Recognition speech-recognition +1

UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning

no code implementations ICCV 2023 Weikang Wan, Haoran Geng, Yun Liu, Zikang Shan, Yaodong Yang, Li Yi, He Wang

We propose a novel, object-agnostic method for learning a universal policy for dexterous object grasping from realistic point cloud observations and proprioceptive information under a table-top setting, namely UniDexGrasp++.

Object

Delving into Discrete Normalizing Flows on SO(3) Manifold for Probabilistic Rotation Modeling

1 code implementation CVPR 2023 Yulin Liu, Haoran Liu, Yingda Yin, Yang Wang, Baoquan Chen, He Wang

Normalizing flows (NFs) provide a powerful tool to construct an expressive distribution by a sequence of trackable transformations of a base distribution and form a probabilistic model of underlying data.

Privacy-preserving Adversarial Facial Features

no code implementations CVPR 2023 Zhibo Wang, He Wang, Shuaifan Jin, Wenwen Zhang, Jiahui Hu, Yan Wang, Peng Sun, Wei Yuan, Kaixin Liu, Kui Ren

In this paper, we propose an adversarial features-based face privacy protection (AdvFace) approach to generate privacy-preserving adversarial features, which can disrupt the mapping from adversarial features to facial images to defend against reconstruction attacks.

Face Recognition Privacy Preserving

Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples

1 code implementation16 May 2023 Wan Jiang, Yunfeng Diao, He Wang, Jianxin Sun, Meng Wang, Richang Hong

Unfortunately, we find UEs provide a false sense of security, because they cannot stop unauthorized users from utilizing other unprotected data to remove the protection, by turning unlearnable data into learnable again.

Towards Robust Probabilistic Modeling on SO(3) via Rotation Laplace Distribution

no code implementations17 May 2023 Yingda Yin, Jiangran Lyu, Yang Wang, He Wang, Baoquan Chen

With this benefit, we demonstrate its advantages in semi-supervised rotation regression, where the pseudo labels are noisy.

regression

RSMT: Real-time Stylized Motion Transition for Characters

1 code implementation21 Jun 2023 Xiangjun Tang, Linjun Wu, He Wang, Bo Hu, Xu Gong, Yuchen Liao, Songnan Li, Qilong Kou, Xiaogang Jin

Styled online in-between motion generation has important application scenarios in computer animation and games.

Human Trajectory Forecasting with Explainable Behavioral Uncertainty

no code implementations4 Jul 2023 Jiangbei Yue, Dinesh Manocha, He Wang

Model-free methods offer superior prediction accuracy but lack explainability, while model-based methods provide explainability but cannot predict well.

Self-Driving Cars Trajectory Forecasting

Understanding the Efficacy of U-Net & Vision Transformer for Groundwater Numerical Modelling

no code implementations8 Jul 2023 Maria Luisa Taccari, Oded Ovadia, He Wang, Adar Kahana, Xiaohui Chen, Peter K. Jimack

This paper presents a comprehensive comparison of various machine learning models, namely U-Net, U-Net integrated with Vision Transformers (ViT), and Fourier Neural Operator (FNO), for time-dependent forward modelling in groundwater systems.

Adaptive Local Basis Functions for Shape Completion

1 code implementation17 Jul 2023 Hui Ying, Tianjia Shao, He Wang, Yin Yang, Kun Zhou

Quantitative and qualitative experiments demonstrate that our method outperforms the state-of-the-art methods in shape completion, detail preservation, generalization to unseen geometries, and computational cost.

Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient

1 code implementation ICCV 2023 Zhengzhi Lu, He Wang, Ziyi Chang, Guoan Yang, Hubert P. H. Shum

Specifically, we first learn a motion manifold where we define an adversarial loss to compute a new gradient for the attack, named skeleton-motion-informed (SMI) gradient.

Action Recognition Adversarial Attack +2

MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction

no code implementations17 Aug 2023 Yijie Tang, Jiazhao Zhang, Zhinan Yu, He Wang, Kai Xu

For the first time, randomized optimization is made possible in neural tracking with several key designs to the learning process, enabling efficient and robust tracking even under fast camera motions.

RGB-D Reconstruction

Few-Shot Physically-Aware Articulated Mesh Generation via Hierarchical Deformation

1 code implementation ICCV 2023 Xueyi Liu, Bin Wang, He Wang, Li Yi

By observing an articulated object dataset containing only a few examples, we wish to learn a model that can generate diverse meshes with high visual fidelity and physical validity.

Philosophy

A Locality-based Neural Solver for Optical Motion Capture

1 code implementation1 Sep 2023 Xiaoyu Pan, Bowen Zheng, Xinwei Jiang, Guanglong Xu, Xianli Gu, Jingxiang Li, Qilong Kou, He Wang, Tianjia Shao, Kun Zhou, Xiaogang Jin

Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking.

Data Augmentation Representation Learning

GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion

no code implementations27 Sep 2023 Jiazhao Zhang, Nandiraju Gireesh, Jilong Wang, Xiaomeng Fang, Chaoyi Xu, Weiguang Chen, Liu Dai, He Wang

Mobile manipulation constitutes a fundamental task for robotic assistants and garners significant attention within the robotics community.

FG-NeRF: Flow-GAN based Probabilistic Neural Radiance Field for Independence-Assumption-Free Uncertainty Estimation

no code implementations28 Sep 2023 Songlin Wei, Jiazhao Zhang, Yang Wang, Fanbo Xiang, Hao Su, He Wang

Existing works rely on the independence assumption of points in the radiance field or the pixels in input views to obtain tractable forms of the probability density function.

Reconstructing 3D Human Pose from RGB-D Data with Occlusions

no code implementations2 Oct 2023 Bowen Dang, Xi Zhao, BoWen Zhang, He Wang

Our key idea is to constrain the solution space of the human body by considering the occluded body parts and visible body parts separately: modeling all plausible poses where the occluded body parts do not penetrate the scene, and constraining the visible body parts using depth data.

STOPNet: Multiview-based 6-DoF Suction Detection for Transparent Objects on Production Lines

no code implementations9 Oct 2023 Yuxuan Kuang, Qin Han, Danshi Li, Qiyu Dai, Lian Ding, Dong Sun, Hanlin Zhao, He Wang

In this work, we present STOPNet, a framework for 6-DoF object suction detection on production lines, with a focus on but not limited to transparent objects, which is an important and challenging problem in robotic systems and modern industry.

Transparent objects

Affective and Dynamic Beam Search for Story Generation

1 code implementation23 Oct 2023 Tenghao Huang, Ehsan Qasemi, Bangzheng Li, He Wang, Faeze Brahman, Muhao Chen, Snigdha Chaturvedi

Storytelling's captivating potential makes it a fascinating research area, with implications for entertainment, education, therapy, and cognitive studies.

Sentence Story Generation

Make a Donut: Hierarchical EMD-Space Planning for Zero-Shot Deformable Manipulation with Tools

no code implementations5 Nov 2023 Yang You, Bokui Shen, Congyue Deng, Haoran Geng, Songlin Wei, He Wang, Leonidas Guibas

Remarkably, our model demonstrates robust generalization capabilities to novel and previously unencountered complex tasks without any preliminary demonstrations.

Deformable Object Manipulation Model Predictive Control

RoboGPT: an intelligent agent of making embodied long-term decisions for daily instruction tasks

no code implementations27 Nov 2023 Yaran Chen, Wenbo Cui, Yuanwen Chen, Mining Tan, Xinyao Zhang, Dongbin Zhao, He Wang

To address the problem, we propose a RoboGPT agent\footnote{our code and dataset will be released soon} for making embodied long-term decisions for daily tasks, with two modules: 1) LLMs-based planning with re-plan to break the task into multiple sub-goals; 2) RoboSkill individually designed for sub-goals to learn better navigation and manipulation skills.

Common Sense Reasoning

Communication-Efficient Federated Optimization over Semi-Decentralized Networks

no code implementations30 Nov 2023 He Wang, Yuejie Chi

In large-scale federated and decentralized learning, communication efficiency is one of the most challenging bottlenecks.

SAGE: Bridging Semantic and Actionable Parts for GEneralizable Articulated-Object Manipulation under Language Instructions

no code implementations3 Dec 2023 Haoran Geng, Songlin Wei, Congyue Deng, Bokui Shen, He Wang, Leonidas Guibas

To address this problem, we propose SAGE, a novel framework that bridges the understanding of semantic and actionable parts of articulated objects to achieve generalizable manipulation under language instructions.

Object

MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition

no code implementations7 Jan 2024 He Wang, Pengcheng Guo, Pan Zhou, Lei Xie

While automatic speech recognition (ASR) systems degrade significantly in noisy environments, audio-visual speech recognition (AVSR) systems aim to complement the audio stream with noise-invariant visual cues and improve the system's robustness.

Audio-Visual Speech Recognition Automatic Speech Recognition +4

ICMC-ASR: The ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition Challenge

no code implementations7 Jan 2024 He Wang, Pengcheng Guo, Yue Li, Ao Zhang, Jiayao Sun, Lei Xie, Wei Chen, Pan Zhou, Hui Bu, Xin Xu, BinBin Zhang, Zhuo Chen, Jian Wu, Longbiao Wang, Eng Siong Chng, Sun Li

To promote speech processing and recognition research in driving scenarios, we build on the success of the Intelligent Cockpit Speech Recognition Challenge (ICSRC) held at ISCSLP 2022 and launch the ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition (ICMC-ASR) Challenge.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023

2 code implementations7 Jan 2024 He Wang, Pengcheng Guo, Wei Chen, Pan Zhou, Lei Xie

This paper delineates the visual speech recognition (VSR) system introduced by the NPU-ASLP-LiAuto (Team 237) in the first Chinese Continuous Visual Speech Recognition Challenge (CNVSRC) 2023, engaging in the fixed and open tracks of Single-Speaker VSR Task, and the open track of Multi-Speaker VSR Task.

speech-recognition Visual Speech Recognition

Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems

no code implementations8 Jan 2024 Biao Yuan, Ana Heitor, He Wang, Xiaohui Chen

Meanwhile, the loss functions for different cases are introduced, and their differences in three-dimensional consolidation problems are highlighted.

MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation

no code implementations15 Jan 2024 Mi Yan, Jiazhao Zhang, Yan Zhu, He Wang

The corresponding 3D points cluster of these 2D mask clusters can be regarded as 3D instances, along with the fused open-vocabulary features from clustered 2D masks.

3D Instance Segmentation 3D Open-Vocabulary Instance Segmentation +6

PIP-Net: Pedestrian Intention Prediction in the Wild

no code implementations20 Feb 2024 Mohsen Azarmi, Mahdi Rezaei, He Wang, Sebastien Glaser

Accurate pedestrian intention prediction (PIP) by Autonomous Vehicles (AVs) is one of the current research challenges in this field.

Autonomous Vehicles

NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

no code implementations24 Feb 2024 Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, He Wang

Vision-and-Language Navigation (VLN) stands as a key research problem of Embodied AI, aiming at enabling agents to navigate in unseen environments following linguistic instructions.

Decision Making Instruction Following +3

SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents

no code implementations27 Feb 2024 Wei Xiang, Haoteng Yin, He Wang, Xiaogang Jin

Pedestrian trajectory prediction is the key technology in many applications for providing insights into human behavior and anticipating human future motions.

Pedestrian Trajectory Prediction Trajectory Prediction

ShapeLLM: Universal 3D Object Understanding for Embodied Interaction

3 code implementations27 Feb 2024 Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, He Wang, Li Yi, Kaisheng Ma

This paper presents ShapeLLM, the first 3D Multimodal Large Language Model (LLM) designed for embodied interaction, exploring a universal 3D object understanding with 3D point clouds and languages.

3D Point Cloud Linear Classification 3D Question Answering (3D-QA) +8

Bayesian Differentiable Physics for Cloth Digitalization

1 code implementation27 Feb 2024 Deshan Gong, Ningtao Mao, He Wang

To learn from small data, we propose a new Bayesian differentiable cloth model to estimate the complex material heterogeneity of real cloths.

Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processes

no code implementations19 Mar 2024 He Wang, Laixi Shi, Yuejie Chi

In offline reinforcement learning (RL), the absence of active exploration calls for attention on the model robustness to tackle the sim-to-real gap, where the discrepancy between the simulated and deployed environments can significantly undermine the performance of the learned policy.

Reinforcement Learning (RL)

Human Motion Prediction under Unexpected Perturbation

no code implementations23 Mar 2024 Jiangbei Yue, Baiyi Li, Julien Pettré, Armin Seyfried, He Wang

We investigate a new task in human motion prediction, which is predicting motions under unexpected physical perturbation potentially involving multiple people.

Human motion prediction motion prediction

A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output

1 code implementation LREC 2022 Vivien Macketanz, Eleftherios Avramidis, Aljoscha Burchardt, He Wang, Renlong Ai, Shushen Manakhimova, Ursula Strohriegel, Sebastian Möller, Hans Uszkoreit

Furthermore, we present various exemplary applications of our test suite that have been implemented in the past years, like contributions to the Conference of Machine Translation, the usage of the test suite and MT outputs for quality estimation, and the expansion of the test suite to the language pair Portuguese–English.

Machine Translation

Cannot find the paper you are looking for? You can Submit a new open access paper.