Search Results for author: He Wang

Found 151 papers, 67 papers with code

A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output

1 code implementation LREC 2022 Vivien Macketanz, Eleftherios Avramidis, Aljoscha Burchardt, He Wang, Renlong Ai, Shushen Manakhimova, Ursula Strohriegel, Sebastian Möller, Hans Uszkoreit

Furthermore, we present various exemplary applications of our test suite that have been implemented in the past years, like contributions to the Conference of Machine Translation, the usage of the test suite and MT outputs for quality estimation, and the expansion of the test suite to the language pair Portuguese–English.

Machine Translation

Generative modeling of Sparse Approximate Inverse Preconditioners

no code implementations17 May 2024 Mou Li, He Wang, Peter K. Jimack

We present a new deep learning paradigm for the generation of sparse approximate inverse (SPAI) preconditioners for matrix systems arising from the mesh-based discretization of elliptic differential operators.

The distributed biased min-consensus protocol revisited: pre-specified finite time control strategies and small-gain based analysis

no code implementations14 May 2024 Yuanqiu Mo, He Wang

Unlike the classical distributed consensus protocols enabling the group of agents as a whole to reach an agreement regarding a certain quantity of interest in a distributed fashion, the distributed biased min-consensus protocol (DBMC) has been proven to generate advanced complexity pertaining to solving the shortest path problem.

ASGrasp: Generalizable Transparent Object Reconstruction and Grasping from RGB-D Active Stereo Camera

no code implementations9 May 2024 Jun Shi, Yong A, Yixiang Jin, Dingzhe Li, Haoyu Niu, Zhezhu Jin, He Wang

ASGrasp utilizes a two-layer learning-based stereo network for the purpose of transparent object reconstruction, enabling material-agnostic object grasping in cluttered environments.

Object Object Reconstruction

Sample Design Engineering: An Empirical Study of What Makes Good Downstream Fine-Tuning Samples for LLMs

1 code implementation19 Apr 2024 Biyang Guo, He Wang, Wenyilin Xiao, Hong Chen, Zhuxin Lee, Songqiao Han, Hailiang Huang

In the burgeoning field of Large Language Models (LLMs) like ChatGPT and LLaMA, Prompt Engineering (PE) is renowned for boosting zero-shot or in-context learning (ICL) through prompt modifications.

Event Extraction In-Context Learning +2

Investigating Guiding Information for Adaptive Collocation Point Sampling in PINNs

no code implementations18 Apr 2024 Jose Florido, He Wang, Amirul Khan, Peter K. Jimack

Physics-informed neural networks (PINNs) provide a means of obtaining approximate solutions of partial differential equations and systems through the minimisation of an objective function which includes the evaluation of a residual function at a set of collocation points within the domain.

Enhancing Lip Reading with Multi-Scale Video and Multi-Encoder

no code implementations8 Apr 2024 He Wang, Pengcheng Guo, Xucheng Wan, Huan Zhou, Lei Xie

Automatic lip-reading (ALR) aims to automatically transcribe spoken content from a speaker's silent lip motion captured in video.

Lipreading Lip Reading +1

Two-Person Interaction Augmentation with Skeleton Priors

no code implementations8 Apr 2024 Baiyi Li, Edmond S. L. Ho, Hubert P. H. Shum, He Wang

Close and continuous interaction with rich contacts is a crucial aspect of human activities (e. g. hugging, dancing) and of interest in many domains like activity recognition, motion prediction, character animation, etc.

Activity Recognition motion prediction

Human Motion Prediction under Unexpected Perturbation

no code implementations CVPR 2024 Jiangbei Yue, Baiyi Li, Julien Pettré, Armin Seyfried, He Wang

We investigate a new task in human motion prediction, which is predicting motions under unexpected physical perturbation potentially involving multiple people.

Human motion prediction motion prediction

Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processes

no code implementations19 Mar 2024 He Wang, Laixi Shi, Yuejie Chi

In offline reinforcement learning (RL), the absence of active exploration calls for attention on the model robustness to tackle the sim-to-real gap, where the discrepancy between the simulated and deployed environments can significantly undermine the performance of the learned policy.

Reinforcement Learning (RL)

Bayesian Differentiable Physics for Cloth Digitalization

1 code implementation CVPR 2024 Deshan Gong, Ningtao Mao, He Wang

To learn from small data, we propose a new Bayesian differentiable cloth model to estimate the complex material heterogeneity of real cloths.

ShapeLLM: Universal 3D Object Understanding for Embodied Interaction

3 code implementations27 Feb 2024 Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, He Wang, Li Yi, Kaisheng Ma

This paper presents ShapeLLM, the first 3D Multimodal Large Language Model (LLM) designed for embodied interaction, exploring a universal 3D object understanding with 3D point clouds and languages.

3D Object Captioning 3D Point Cloud Linear Classification +10

SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents

1 code implementation27 Feb 2024 Wei Xiang, Haoteng Yin, He Wang, Xiaogang Jin

Pedestrian trajectory prediction is the key technology in many applications for providing insights into human behavior and anticipating human future motions.

Pedestrian Trajectory Prediction Trajectory Prediction

NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

no code implementations24 Feb 2024 Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, He Wang

Vision-and-language navigation (VLN) stands as a key research problem of Embodied AI, aiming at enabling agents to navigate in unseen environments following linguistic instructions.

Decision Making Instruction Following +3

PIP-Net: Pedestrian Intention Prediction in the Wild

no code implementations20 Feb 2024 Mohsen Azarmi, Mahdi Rezaei, He Wang, Sebastien Glaser

Accurate pedestrian intention prediction (PIP) by Autonomous Vehicles (AVs) is one of the current research challenges in this field.

Autonomous Vehicles

Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems

no code implementations8 Jan 2024 Biao Yuan, Ana Heitor, He Wang, Xiaohui Chen

Meanwhile, the loss functions for different cases are introduced, and their differences in three-dimensional consolidation problems are highlighted.

The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023

2 code implementations7 Jan 2024 He Wang, Pengcheng Guo, Wei Chen, Pan Zhou, Lei Xie

This paper delineates the visual speech recognition (VSR) system introduced by the NPU-ASLP-LiAuto (Team 237) in the first Chinese Continuous Visual Speech Recognition Challenge (CNVSRC) 2023, engaging in the fixed and open tracks of Single-Speaker VSR Task, and the open track of Multi-Speaker VSR Task.

Decoder speech-recognition +1

MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition

no code implementations7 Jan 2024 He Wang, Pengcheng Guo, Pan Zhou, Lei Xie

While automatic speech recognition (ASR) systems degrade significantly in noisy environments, audio-visual speech recognition (AVSR) systems aim to complement the audio stream with noise-invariant visual cues and improve the system's robustness.

Audio-Visual Speech Recognition Automatic Speech Recognition +4

ICMC-ASR: The ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition Challenge

no code implementations7 Jan 2024 He Wang, Pengcheng Guo, Yue Li, Ao Zhang, Jiayao Sun, Lei Xie, Wei Chen, Pan Zhou, Hui Bu, Xin Xu, BinBin Zhang, Zhuo Chen, Jian Wu, Longbiao Wang, Eng Siong Chng, Sun Li

To promote speech processing and recognition research in driving scenarios, we build on the success of the Intelligent Cockpit Speech Recognition Challenge (ICSRC) held at ISCSLP 2022 and launch the ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition (ICMC-ASR) Challenge.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

SAGE: Bridging Semantic and Actionable Parts for GEneralizable Manipulation of Articulated Objects

no code implementations3 Dec 2023 Haoran Geng, Songlin Wei, Congyue Deng, Bokui Shen, He Wang, Leonidas Guibas

More concretely, given an articulated object, we first observe all the semantic parts on it, conditioned on which an instruction interpreter proposes possible action programs that concretize the natural language instruction.

Language Modelling Object

Communication-Efficient Federated Optimization over Semi-Decentralized Networks

no code implementations30 Nov 2023 He Wang, Yuejie Chi

In large-scale federated and decentralized learning, communication efficiency is one of the most challenging bottlenecks.

RoboGPT: an intelligent agent of making embodied long-term decisions for daily instruction tasks

no code implementations27 Nov 2023 Yaran Chen, Wenbo Cui, Yuanwen Chen, Mining Tan, Xinyao Zhang, Dongbin Zhao, He Wang

To address the problem, we propose a RoboGPT agent\footnote{our code and dataset will be released soon} for making embodied long-term decisions for daily tasks, with two modules: 1) LLMs-based planning with re-plan to break the task into multiple sub-goals; 2) RoboSkill individually designed for sub-goals to learn better navigation and manipulation skills.

Common Sense Reasoning

Make a Donut: Hierarchical EMD-Space Planning for Zero-Shot Deformable Manipulation with Tools

no code implementations5 Nov 2023 Yang You, Bokui Shen, Congyue Deng, Haoran Geng, Songlin Wei, He Wang, Leonidas Guibas

Remarkably, our model demonstrates robust generalization capabilities to novel and previously unencountered complex tasks without any preliminary demonstrations.

Deformable Object Manipulation Model Predictive Control

Affective and Dynamic Beam Search for Story Generation

1 code implementation23 Oct 2023 Tenghao Huang, Ehsan Qasemi, Bangzheng Li, He Wang, Faeze Brahman, Muhao Chen, Snigdha Chaturvedi

Storytelling's captivating potential makes it a fascinating research area, with implications for entertainment, education, therapy, and cognitive studies.

Sentence Story Generation

STOPNet: Multiview-based 6-DoF Suction Detection for Transparent Objects on Production Lines

no code implementations9 Oct 2023 Yuxuan Kuang, Qin Han, Danshi Li, Qiyu Dai, Lian Ding, Dong Sun, Hanlin Zhao, He Wang

In this work, we present STOPNet, a framework for 6-DoF object suction detection on production lines, with a focus on but not limited to transparent objects, which is an important and challenging problem in robotic systems and modern industry.

Transparent objects

Reconstructing 3D Human Pose from RGB-D Data with Occlusions

no code implementations2 Oct 2023 Bowen Dang, Xi Zhao, BoWen Zhang, He Wang

Our key idea is to constrain the solution space of the human body by considering the occluded body parts and visible body parts separately: modeling all plausible poses where the occluded body parts do not penetrate the scene, and constraining the visible body parts using depth data.

FG-NeRF: Flow-GAN based Probabilistic Neural Radiance Field for Independence-Assumption-Free Uncertainty Estimation

no code implementations28 Sep 2023 Songlin Wei, Jiazhao Zhang, Yang Wang, Fanbo Xiang, Hao Su, He Wang

Existing works rely on the independence assumption of points in the radiance field or the pixels in input views to obtain tractable forms of the probability density function.

GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion

no code implementations27 Sep 2023 Jiazhao Zhang, Nandiraju Gireesh, Jilong Wang, Xiaomeng Fang, Chaoyi Xu, Weiguang Chen, Liu Dai, He Wang

Mobile manipulation constitutes a fundamental task for robotic assistants and garners significant attention within the robotics community.

A Locality-based Neural Solver for Optical Motion Capture

1 code implementation1 Sep 2023 Xiaoyu Pan, Bowen Zheng, Xinwei Jiang, Guanglong Xu, Xianli Gu, Jingxiang Li, Qilong Kou, He Wang, Tianjia Shao, Kun Zhou, Xiaogang Jin

Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking.

Data Augmentation Graph Neural Network +1

Few-Shot Physically-Aware Articulated Mesh Generation via Hierarchical Deformation

1 code implementation ICCV 2023 Xueyi Liu, Bin Wang, He Wang, Li Yi

By observing an articulated object dataset containing only a few examples, we wish to learn a model that can generate diverse meshes with high visual fidelity and physical validity.


MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction

no code implementations17 Aug 2023 Yijie Tang, Jiazhao Zhang, Zhinan Yu, He Wang, Kai Xu

For the first time, randomized optimization is made possible in neural tracking with several key designs to the learning process, enabling efficient and robust tracking even under fast camera motions.

RGB-D Reconstruction

Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient

1 code implementation ICCV 2023 Zhengzhi Lu, He Wang, Ziyi Chang, Guoan Yang, Hubert P. H. Shum

Specifically, we first learn a motion manifold where we define an adversarial loss to compute a new gradient for the attack, named skeleton-motion-informed (SMI) gradient.

Action Recognition Adversarial Attack +2

Adaptive Local Basis Functions for Shape Completion

1 code implementation17 Jul 2023 Hui Ying, Tianjia Shao, He Wang, Yin Yang, Kun Zhou

Quantitative and qualitative experiments demonstrate that our method outperforms the state-of-the-art methods in shape completion, detail preservation, generalization to unseen geometries, and computational cost.

Understanding the Efficacy of U-Net & Vision Transformer for Groundwater Numerical Modelling

no code implementations8 Jul 2023 Maria Luisa Taccari, Oded Ovadia, He Wang, Adar Kahana, Xiaohui Chen, Peter K. Jimack

This paper presents a comprehensive comparison of various machine learning models, namely U-Net, U-Net integrated with Vision Transformers (ViT), and Fourier Neural Operator (FNO), for time-dependent forward modelling in groundwater systems.

Human Trajectory Forecasting with Explainable Behavioral Uncertainty

no code implementations4 Jul 2023 Jiangbei Yue, Dinesh Manocha, He Wang

Model-free methods offer superior prediction accuracy but lack explainability, while model-based methods provide explainability but cannot predict well.

Self-Driving Cars Trajectory Forecasting

RSMT: Real-time Stylized Motion Transition for Characters

1 code implementation21 Jun 2023 Xiangjun Tang, Linjun Wu, He Wang, Bo Hu, Xu Gong, Yuchen Liao, Songnan Li, Qilong Kou, Xiaogang Jin

Styled online in-between motion generation has important application scenarios in computer animation and games.

Towards Robust Probabilistic Modeling on SO(3) via Rotation Laplace Distribution

no code implementations17 May 2023 Yingda Yin, Jiangran Lyu, Yang Wang, He Wang, Baoquan Chen

With this benefit, we demonstrate its advantages in semi-supervised rotation regression, where the pseudo labels are noisy.


Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples

1 code implementation16 May 2023 Wan Jiang, Yunfeng Diao, He Wang, Jianxin Sun, Meng Wang, Richang Hong

Unfortunately, we find UEs provide a false sense of security, because they cannot stop unauthorized users from utilizing other unprotected data to remove the protection, by turning unlearnable data into learnable again.

Privacy-preserving Adversarial Facial Features

no code implementations CVPR 2023 Zhibo Wang, He Wang, Shuaifan Jin, Wenwen Zhang, Jiahui Hu, Yan Wang, Peng Sun, Wei Yuan, Kaixin Liu, Kui Ren

In this paper, we propose an adversarial features-based face privacy protection (AdvFace) approach to generate privacy-preserving adversarial features, which can disrupt the mapping from adversarial features to facial images to defend against reconstruction attacks.

Face Recognition Privacy Preserving

Delving into Discrete Normalizing Flows on SO(3) Manifold for Probabilistic Rotation Modeling

1 code implementation CVPR 2023 Yulin Liu, Haoran Liu, Yingda Yin, Yang Wang, Baoquan Chen, He Wang

Normalizing flows (NFs) provide a powerful tool to construct an expressive distribution by a sequence of trackable transformations of a base distribution and form a probabilistic model of underlying data.

UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning

no code implementations ICCV 2023 Weikang Wan, Haoran Geng, Yun Liu, Zikang Shan, Yaodong Yang, Li Yi, He Wang

We propose a novel, object-agnostic method for learning a universal policy for dexterous object grasping from realistic point cloud observations and proprioceptive information under a table-top setting, namely UniDexGrasp++.


The NPU-ASLP System for Audio-Visual Speech Recognition in MISP 2022 Challenge

no code implementations11 Mar 2023 Pengcheng Guo, He Wang, Bingshen Mu, Ao Zhang, Peikun Chen

This paper describes our NPU-ASLP system for the Audio-Visual Diarization and Recognition (AVDR) task in the Multi-modal Information based Speech Processing (MISP) 2022 Challenge.

Audio-Visual Speech Recognition speech-recognition +1

A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimation

no code implementations3 Mar 2023 Yingda Yin, Yang Wang, He Wang, Baoquan Chen

Rotation Laplace distribution is robust to the disturbance of outliers and enforces much gradient to the low-error region, resulting in a better convergence.


UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy

1 code implementation CVPR 2023 Yinzhen Xu, Weikang Wan, Jialiang Zhang, Haoran Liu, Zikang Shan, Hao Shen, Ruicheng Wang, Haoran Geng, Yijia Weng, Jiayi Chen, Tengyu Liu, Li Yi, He Wang

Trained on our synthesized large-scale dexterous grasp dataset, this model enables us to sample diverse and high-quality dexterous grasp poses for the object point cloud. For the second stage, we propose to replace the motion planning used in parallel gripper grasping with a goal-conditioned grasp policy, due to the complexity involved in dexterous grasping execution.

Motion Planning

Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance

1 code implementation28 Feb 2023 Xueyi Liu, Ji Zhang, Ruizhen Hu, Haibin Huang, He Wang, Li Yi

Category-level articulated object pose estimation aims to estimate a hierarchy of articulation-aware object poses of an unseen articulated object from a known category.

Disentanglement Object +1

Adaptive Zone-Aware Hierarchical Planner for Vision-Language Navigation

1 code implementation CVPR 2023 Chen Gao, Xingyu Peng, Mi Yan, He Wang, Lirong Yang, Haibing Ren, Hongsheng Li, Si Liu

In this paper, we propose an Adaptive Zone-aware Hierarchical Planner (AZHP) to explicitly divides the navigation process into two heterogeneous phases, i. e., sub-goal setting via zone partition/selection (high-level action) and sub-goal executing (low-level action), for hierarchical planning.

Vision-Language Navigation

Full-Body Articulated Human-Object Interaction

1 code implementation ICCV 2023 Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, Siyuan Huang

By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation to tackle the estimation of articulated object poses and shapes during whole-body interactions.

Action Recognition Human-Object Interaction Detection +3

3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification

no code implementations CVPR 2023 Jiazhao Zhang, Liu Dai, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, He Wang

However, leveraging 3D scene representation can be prohibitively unpractical for policy learning in this floor-level task, due to low sample efficiency and expensive computational cost.

Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack

4 code implementations21 Nov 2022 Yunfeng Diao, He Wang, Tianjia Shao, Yong-Liang Yang, Kun Zhou, David Hogg, Meng Wang

Via BASAR, we find on-manifold adversarial samples are extremely deceitful and rather common in skeletal motions, in contrast to the common belief that adversarial samples only exist off-manifold.

Adversarial Attack Human Activity Recognition +2

LeNo: Adversarial Robust Salient Object Detection Networks with Learnable Noise

1 code implementation27 Oct 2022 He Wang, Lin Wan, He Tang

In general, LeNo consists of a simple shallow noise and noise estimation that embedded in the encoder and decoder of arbitrary SOD networks respectively.

Decoder Noise Estimation +4

GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF

1 code implementation12 Oct 2022 Qiyu Dai, Yan Zhu, Yiran Geng, Ciyu Ruan, Jiazhao Zhang, He Wang

In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry.

Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing

1 code implementation8 Oct 2022 Yun Liu, Xiaomeng Xu, Weihang Chen, Haocheng Yuan, He Wang, Jing Xu, Rui Chen, Li Yi

When manipulating an object to accomplish complex tasks, humans rely on both vision and touch to keep track of the object's 6D pose.

hand-object pose Object +1

DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation

no code implementations6 Oct 2022 Ruicheng Wang, Jialiang Zhang, Jiayi Chen, Yinzhen Xu, Puhao Li, Tengyu Liu, He Wang

Robotic dexterous grasping is the first step to enable human-like dexterous object manipulation and thus a crucial robotic technology.


Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild

no code implementations24 Sep 2022 Jiayi Chen, Mi Yan, Jiazhao Zhang, Yinzhen Xu, Xiaolong Li, Yijia Weng, Li Yi, Shuran Song, He Wang

We for the first time propose a point cloud based hand joint tracking network, HandTrackNet, to estimate the inter-frame hand joint motion.

hand-object pose Object +2

Shape Completion with Points in the Shadow

1 code implementation17 Sep 2022 BoWen Zhang, Xi Zhao, He Wang, Ruizhen Hu

The core challenge is to generate plausible geometries to fill the unobserved part of the object based on a partial scan, which is under-constrained and suffers from a huge solution space.

Object Point Cloud Completion

Talking Head from Speech Audio using a Pre-trained Image Generator

no code implementations9 Sep 2022 Mohammed M. Alghamdi, He Wang, Andrew J. Bulpitt, David C. Hogg

We train a recurrent neural network to map from speech utterances to displacements in the latent space of the image generator.


Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects

1 code implementation7 Aug 2022 Qiyu Dai, Jiyao Zhang, Qiwei Li, Tianhao Wu, Hao Dong, Ziyuan Liu, Ping Tan, He Wang

Commercial depth sensors usually generate noisy and missing depths, especially on specular and transparent objects, which poses critical issues to downstream depth or point cloud-based tasks.

Pose Estimation Transparent objects

Human Trajectory Prediction via Neural Social Physics

1 code implementation21 Jul 2022 Jiangbei Yue, Dinesh Manocha, He Wang

Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters.

Inductive Bias Trajectory Prediction

Space-based gravitational wave signal detection and extraction with deep neural network

1 code implementation15 Jul 2022 Tianyu Zhao, Ruoxi Lyu, He Wang, Zhoujian Cao, Zhixiang Ren

Space-based gravitational wave (GW) detectors will be able to observe signals from sources that are otherwise nearly impossible from current ground-based detection.

Underdetermined 2D-DOD and 2D-DOA Estimation for Bistatic Coprime EMVS-MIMO Radar: From the Difference Coarray Perspective

no code implementations6 Jun 2022 Qianpeng Xie, Yihang Du, He Wang, Xiaoyi Pan, Feng Zhao

Firstly, a 5-D tensor model was constructed by using the multi-dimensional space-time characteristics of the received data.

8D Parameters Estimation for Bistatic EMVS-MIMO Radar via the nested PARAFAC

no code implementations4 Jun 2022 Qianpeng Xie, He Wang, Yihang Du, Xiaoyi Pan, Feng Zhao

Firstly, the outer part PARAFAC algorithm was carried out to estimate the receive spatial response matrix and its first way factor matrix.

Real-time Controllable Motion Transition for Characters

no code implementations5 May 2022 Xiangjun Tang, He Wang, Bo Hu, Xu Gong, Ruifan Yi, Qilong Kou, Xiaogang Jin

Then, during generation, we design a transition model which is essentially a sampling strategy to sample from the learned manifold, based on the target frame and the aimed transition duration.

Attention U-Net as a surrogate model for groundwater prediction

no code implementations9 Apr 2022 Maria Luisa Taccari, Jonathan Nuttall, Xiaohui Chen, He Wang, Bennie Minnema, Peter K. Jimack

This manuscript presents an Attention U-Net model that attempts to capture the fundamental input-output relations of the groundwater system and generates solutions of hydraulic head in the whole domain given a set of physical parameters and boundary conditions.


Multi-Robot Active Mapping via Neural Bipartite Graph Matching

no code implementations CVPR 2022 Kai Ye, Siyan Dong, Qingnan Fan, He Wang, Li Yi, Fei Xia, Jue Wang, Baoquan Chen

Previous approaches either choose the frontier as the goal position via a myopic solution that hinders the time efficiency, or maximize the long-term value via reinforcement learning to directly regress the goal position, but does not guarantee the complete map construction.

Graph Matching Graph Neural Network +3

FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering

no code implementations CVPR 2022 Yingda Yin, Yingcheng Cai, He Wang, Baoquan Chen

Inspired by the popular semi-supervised approach, FixMatch, we propose to leverage pseudo label filtering to facilitate the information flow from labeled data to unlabeled data in a teacher-student mutual learning framework.

Pseudo Label Pseudo Label Filtering +1

iPLAN: Interactive and Procedural Layout Planning

1 code implementation CVPR 2022 Feixiang He, Yanlong Huang, He Wang

However, the capability of involving humans into the loop has been largely ignored in existing methods which are mostly end-to-end approaches.

Image Generation Layout Design

CodedVTR: Codebook-based Sparse Voxel Transformer with Geometric Guidance

no code implementations CVPR 2022 Tianchen Zhao, Niansong Zhang, Xuefei Ning, He Wang, Li Yi, Yu Wang

We propose CodedVTR (Codebook-based Voxel TRansformer), which improves data efficiency and generalization ability for 3D sparse voxel transformers.

3D Semantic Segmentation

Defending Black-box Skeleton-based Human Activity Classifiers

2 code implementations9 Mar 2022 He Wang, Yunfeng Diao, Zichang Tan, Guodong Guo

Our method is featured by full Bayesian treatments of the clean data, the adversaries and the classifier, leading to (1) a new Bayesian Energy-based formulation of robust discriminative classifiers, (2) a new adversary sampling scheme based on natural motion manifolds, and (3) a new post-train Bayesian strategy for black-box defense.

Human Activity Recognition Time Series Analysis

Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations

2 code implementations4 Mar 2022 Hao Shen, Weikang Wan, He Wang

Generalizable object manipulation skills are critical for intelligent and multi-functional robots to work in real-world complex scenes.

Imitation Learning

HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction

1 code implementation CVPR 2022 Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, Li Yi

We present HOI4D, a large-scale 4D egocentric dataset with rich annotations, to catalyze the research of category-level human-object interaction.

Action Segmentation Benchmarking +6

PartAfford: Part-level Affordance Discovery from 3D Objects

no code implementations28 Feb 2022 Chao Xu, Yixin Chen, He Wang, Song-Chun Zhu, Yixin Zhu, Siyuan Huang

We propose a novel learning framework for PartAfford, which discovers part-level representations by leveraging only the affordance set supervision and geometric primitive regularization, without dense supervision.


Pose Guided Image Generation from Misaligned Sources via Residual Flow Based Correction

no code implementations2 Feb 2022 Jiawei Lu, He Wang, Tianjia Shao, Yin Yang, Kun Zhou

However, as source images are often misaligned due to the large disparities among the camera settings, strong assumptions have been made in the past with respect to the camera(s) or/and the object in interest, limiting the application of such techniques.

Pose-Guided Image Generation

Fine-grained differentiable physics: a yarn-level model for fabrics

1 code implementation ICLR 2022 Deshan Gong, Zhanxing Zhu, Andrew J. Bulpitt, He Wang

To this end, we propose several differentiable forces, whose counterparts in empirical physics are indifferentiable, to facilitate gradient-based learning.

ADeLA: Automatic Dense Labeling With Attention for Viewpoint Shift in Semantic Segmentation

no code implementations CVPR 2022 Hanxiang Ren, Yanchao Yang, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas J. Guibas

We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.

Hallucination Semantic Segmentation +1

Dynamics-aware Adversarial Attack of 3D Sparse Convolution Network

1 code implementation17 Dec 2021 An Tao, Yueqi Duan, He Wang, Ziyi Wu, Pengliang Ji, Haowen Sun, Jie zhou, Jiwen Lu

It results in a serious issue of lagged gradient, making the learned attack at the current step ineffective due to the architecture changes afterward.

3D Classification 3D Semantic Segmentation +2

Domain Adaptation on Point Clouds via Geometry-Aware Implicits

1 code implementation CVPR 2022 Yuefan Shen, Yanchao Yang, Mi Yan, He Wang, Youyi Zheng, Leonidas Guibas

Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.

Autonomous Driving Unsupervised Domain Adaptation

Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation

no code implementations NeurIPS 2021 Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang

Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.

Object Pose Estimation +1

Unsupervised Image Generation with Infinite Generative Adversarial Networks

1 code implementation ICCV 2021 Hui Ying, He Wang, Tianjia Shao, Yin Yang, Kun Zhou

Image generation has been heavily investigated in computer vision, where one core research challenge is to generate images from arbitrarily complex distributions with little supervision.

Image Generation

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

1 code implementation29 Jul 2021 Yanchao Yang, Hanxiang Ren, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas

Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.

Hallucination Inductive Bias +2

Survey of Image Based Graph Neural Networks

no code implementations11 Jun 2021 Usman Nazir, He Wang, Murtaza Taj

In this survey paper, we analyze image based graph neural networks and propose a three-step classification approach.

Classification Superpixels

Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds

no code implementations NeurIPS 2021 Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang

To reduce the huge amount of pose annotations needed for category-level learning, we propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.

Object Pose Estimation +1

Maneuver-based Anchor Trajectory Hypotheses at Roundabouts

1 code implementation22 Apr 2021 Mohamed Hasan, Evangelos Paschalidis, Albert Solernou, He Wang, Gustav Markkula, Richard Romano

Accordingly, our model employs a set of maneuver-specific anchor trajectories that cover the space of possible outcomes at the roundabout.

Decoder motion prediction +1

CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

1 code implementation ICCV 2021 Yijia Weng, He Wang, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas J. Guibas

For the first time, we propose a unified framework that can handle 9DoF pose tracking for novel rigid object instances as well as per-part pose tracking for articulated objects from known categories.

Pose Tracking

Decentralized Statistical Inference with Unrolled Graph Neural Networks

1 code implementation4 Apr 2021 He Wang, Yifei Shen, Ziyuan Wang, Dongsheng Li, Jun Zhang, Khaled B. Letaief, Jie Lu

In this paper, we investigate the decentralized statistical inference problem, where a network of agents cooperatively recover a (structured) vector from private noisy samples without centralized coordination.

Data-Driven Optimization for Atlanta Police Zone Design

no code implementations30 Mar 2021 Shixiang Zhu, He Wang, Yao Xie

By analyzing data before and after the zone redesign, we show that the new design has reduced the response time to high priority 911 calls by 5. 8\% and the imbalance of police workload among different zones by 43\%.

Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack

1 code implementation CVPR 2021 He Wang, Feixiang He, Zhexi Peng, Tianjia Shao, Yong-Liang Yang, Kun Zhou, David Hogg

In this paper, we examine the robustness of state-of-the-art action recognizers against adversarial attack, which has been rarely investigated so far.

Action Recognition Adversarial Attack +4

BASAR:Black-box Attack on Skeletal Action Recognition

1 code implementation CVPR 2021 Yunfeng Diao, Tianjia Shao, Yong-Liang Yang, Kun Zhou, He Wang

The robustness of skeleton-based activity recognizers has been questioned recently, which shows that they are vulnerable to adversarial attacks when the full-knowledge of the recognizer is accessible to the attacker.

Action Recognition Adversarial Attack +1

Enhanced 3D Human Pose Estimation from Videos by using Attention-Based Neural Network with Dilated Convolutions

1 code implementation4 Mar 2021 Ruixu Liu, Ju Shen, He Wang, Chen Chen, Sen-ching Cheung, Vijayan K. Asari

In this work, we show a systematic design (from 2D to 3D) for how conventional networks and other forms of constraints can be incorporated into the attention framework for learning long-range dependencies for the task of pose estimation.

2D Pose Estimation 3D Human Pose Estimation

Distributed Optimization with Coupling Constraints

no code implementations25 Feb 2021 Xuyang Wu, He Wang, Jie Lu

In this paper, we develop a novel distributed algorithm for addressing convex optimization with both nonlinear inequality and linear equality constraints, where the objective function can be a general nonsmooth convex function and all the constraints can be fully coupled.

Distributed Optimization Optimization and Control

High-order Differentiable Autoencoder for Nonlinear Model Reduction

no code implementations19 Feb 2021 Siyuan Shen, Yang Yin, Tianjia Shao, He Wang, Chenfanfu Jiang, Lei Lan, Kun Zhou

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation.

Vocal Bursts Intensity Prediction

In-game Residential Home Planning via Visual Context-aware Global Relation Learning

no code implementations8 Feb 2021 Lijuan Liu, Yin Yang, Yi Yuan, Tianjia Shao, He Wang, Kun Zhou

In this paper, we propose an effective global relation learning algorithm to recommend an appropriate location of a building unit for in-game customization of residential home complex.

Graph Generation Relation

MoDL-QSM: Model-based Deep Learning for Quantitative Susceptibility Mapping

1 code implementation21 Jan 2021 Ruimin Feng, Jiayi Zhao, He Wang, Baofeng Yang, Jie Feng, Yuting Shi, Ming Zhang, Chunlei Liu, Yuyao Zhang, Jie Zhuang, Hongjiang Wei

However, there exists a mismatch between the observed phase and the theoretical forward phase estimated by the susceptibility label.


Systematic electrochemical etching of various metal tips for tunneling spectroscopy and scanning probe microscopy

no code implementations18 Jan 2021 Jiawei Zhang, Pinyuan Wang, Xuao Zhang, Haoran Ji, Jiawei Luo, He Wang, Jian Wang

To ensure the reproducibility of experimental results, the fabrication of tips should be standardized, and a controllable and convenient system should be set up.

Materials Science

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

1 code implementation CVPR 2021 Jiahui Huang, He Wang, Tolga Birdal, Minhyuk Sung, Federica Arrigoni, Shi-Min Hu, Leonidas Guibas

We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds.

Motion Estimation Motion Segmentation +1

Single Image 3D Shape Retrieval via Cross-Modal Instance and Category Contrastive Learning

no code implementations ICCV 2021 Ming-Xian Lin, Jie Yang, He Wang, Yu-Kun Lai, Rongfei Jia, Binqiang Zhao, Lin Gao

Inspired by the great success in recent contrastive learning works on self-supervised representation learning, we propose a novel IBSR pipeline leveraging contrastive learning.

3D Shape Retrieval Contrastive Learning +4

Detection of magnetic gap in the topological surface states of MnBi2Te4

no code implementations31 Dec 2020 Haoran Ji, Yanzhao Liu, He Wang, Jiawei Luo, Jiaheng Li, Hao Li, Yang Wu, Yong Xu, Jian Wang

An essential ingredient to realize these quantum states is the magnetic gap in the topological surface states induced by the out-of-plane ferromagnetism on the surface of MnBi2Te4.

Materials Science

3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection

2 code implementations CVPR 2021 He Wang, Yezhen Cong, Or Litany, Yue Gao, Leonidas J. Guibas

On KITTI, we are the first to demonstrate semi-supervised 3D object detection and our method surpasses a fully supervised baseline from 1. 8% to 7. 6% under different label ratios and categories.

3D Object Detection Autonomous Driving +1

IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration

2 code implementations11 Oct 2020 Ziyi Wu, Yueqi Duan, He Wang, Qingnan Fan, Leonidas J. Guibas

The former aims to recover the surface of point cloud through implicit function, while the latter encourages evenly-distributed points.

Constant Regret Re-solving Heuristics for Price-based Revenue Management

no code implementations7 Sep 2020 Yining Wang, He Wang

First, we prove that a natural re-solving heuristic attains $O(1)$ regret compared to the value of the optimal policy.


Dynamic Future Net: Diversified Human Motion Generation

no code implementations25 Aug 2020 Wenheng Chen, He Wang, Yi Yuan, Tianjia Shao, Kun Zhou

We evaluate our model on a wide range of motions and compare it with the state-of-the-art methods.

Object-Centric Multi-View Aggregation

no code implementations20 Jul 2020 Shubham Tulsiani, Or Litany, Charles R. Qi, He Wang, Leonidas J. Guibas

We present an approach for aggregating a sparse set of views of an object in order to compute a semi-implicit 3D representation in the form of a volumetric feature grid.

Novel View Synthesis Object +1

Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks

no code implementations12 Jun 2020 He Wang, Zetian Jiang, Li Yi, Kaichun Mo, Hao Su, Leonidas J. Guibas

We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics.

Clustering valid

Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance

no code implementations29 Apr 2020 Feixiang He, Yuanhang Xiang, Xi Zhao, He Wang

The method takes as input raw and noisy data with highly mixed multi-dimensional (space, time and dynamics) information, and automatically structure it by learning the correlations among these dimensions.

Data Visualization

MeshingNet: A New Mesh Generation Method based on Deep Learning

no code implementations15 Apr 2020 Zheyan Zhang, Yongxing Wang, Peter K. Jimack, He Wang

The framework that we have developed is based around training an artificial neural network (ANN) to guide standard mesh generation software, based upon a prediction of the required local mesh density throughout the domain.

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

1 code implementation ECCV 2020 Kaichun Mo, He Wang, Xinchen Yan, Leonidas J. Guibas

3D generative shape modeling is a fundamental research area in computer vision and interactive computer graphics, with many real-world applications.

3D Shape Generation

SAPIEN: A SimulAted Part-based Interactive ENvironment

1 code implementation CVPR 2020 Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su

To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.


Curriculum DeepSDF

1 code implementation ECCV 2020 Yueqi Duan, Haidong Zhu, He Wang, Li Yi, Ram Nevatia, Leonidas J. Guibas

When learning to sketch, beginners start with simple and flexible shapes, and then gradually strive for more complex and accurate ones in the subsequent training sessions.

3D Shape Representation Representation Learning

Human-like Planning for Reaching in Cluttered Environments

1 code implementation28 Feb 2020 Mohamed Hasan, Matthew Warburton, Wisdom C. Agboh, Mehmet R. Dogar, Matteo Leonetti, He Wang, Faisal Mushtaq, Mark Mon-Williams, Anthony G. Cohn

From this, we devised a qualitative representation of the task space to abstract the decision making, irrespective of the number of obstacles.

Decision Making

Predicting the Physical Dynamics of Unseen 3D Objects

1 code implementation16 Jan 2020 Davis Rempe, Srinath Sridhar, He Wang, Leonidas J. Guibas

Experiments show that we can accurately predict the changes in state for unseen object geometries and initial conditions.


Category-Level Articulated Object Pose Estimation

2 code implementations CVPR 2020 Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song

We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space.

Object Pose Estimation

SMART: Skeletal Motion Action Recognition aTtack

no code implementations16 Nov 2019 He Wang, Feixiang He, Zhexi Peng, Yong-Liang Yang, Tianjia Shao, Kun Zhou, David Hogg

In this paper, we propose a method, SMART, to attack action recognizers which rely on 3D skeletal motions.

Action Recognition Adversarial Attack +2

Bimodular continuous attractor neural networks with static and moving stimuli

no code implementations16 Oct 2019 Min Yan, Wen-Hao Zhang, He Wang, K. Y. Michael Wong

We found that when bumps coexist in both modules, the position of each bump is shifted towards the other input when the intermodular couplings are excitatory and is shifted away when inhibitory.

Causal Inference

Gravitational wave signal recognition of O1 data by deep learning

1 code implementation30 Sep 2019 He Wang, Zhoujian Cao, Xiaolin Liu, Shichao Wu, Jian-Yang Zhu

Our adjusted convolutional neural network admits comparable accuracy and efficiency of signal recognition as other deep learning works published in the literature.

Instrumentation and Methods for Astrophysics General Relativity and Quantum Cosmology

Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling

no code implementations20 Aug 2019 He Wang, Edmond S. L. Ho, Hubert P. H. Shum, Zhanxing Zhu

In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications.

Denoising Time Series Analysis

Competing Against Equilibria in Zero-Sum Games with Evolving Payoffs

1 code implementation17 Jul 2019 Adrian Rivera Cardoso, Jacob Abernethy, He Wang, Huan Xu

Finding the Nash Equilibrium (NE) of a two player zero-sum game is core to many problems in statistics, optimization, and economics, and for a fixed game matrix this can be easily reduced to solving a linear program.

Large Scale Markov Decision Processes with Changing Rewards

no code implementations NeurIPS 2019 Adrian Rivera Cardoso, He Wang, Huan Xu

We consider Markov Decision Processes (MDPs) where the rewards are unknown and may change in an adversarial manner.

DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark

no code implementations23 Apr 2019 Jianwu Fang, Dingxin Yan, Jiahuan Qiao, Jianru Xue, He Wang, Sen Li

Driver attention prediction is currently becoming the focus in safe driving research community, such as the DR(eye)VE project and newly emerged Berkeley DeepDrive Attention (BDD-A) database in critical situations.

Driver Attention Monitoring

Learning Generalizable Physical Dynamics of 3D Rigid Objects

no code implementations2 Jan 2019 Davis Rempe, Srinath Sridhar, He Wang, Leonidas J. Guibas

In this work, we focus on predicting the dynamics of 3D rigid objects, in particular an object's final resting position and total rotation when subjected to an impulsive force.

Autonomous Vehicles Object +1

Globally Continuous and Non-Markovian Activity Analysis from Videos

no code implementations11 Oct 2018 He Wang, Carol O'Sullivan

By combining these patterns with detailed environment information, we interpret the semantics of activities and report anomalies.

Novelty Detection Meets Collider Physics

no code implementations26 Jul 2018 Jan Hajer, Ying-Ying Li, Tao Liu, He Wang

Complementary to supervised learning, it allows to analyze data model-independently.

Clustering Novelty Detection

The Online Saddle Point Problem and Online Convex Optimization with Knapsacks

no code implementations21 Jun 2018 Adrian Rivera, He Wang, Huan Xu

We relate this problem to the online saddle point problem and establish $O(\sqrt{T})$ regret using a primal-dual algorithm.

“Congruent” and “Opposite” Neurons: Sisters for Multisensory Integration and Segregation

no code implementations NeurIPS 2016 Wen-Hao Zhang, He Wang, K. Y. Michael Wong, Si Wu

Mimicking the experimental protocol, our model reproduces the characteristics of congruent and opposite neurons, and demonstrates that in each module, the sisters of congruent and opposite neurons can jointly achieve optimal multisensory information integration and segregation.

Saliency Detection via Cellular Automata

no code implementations CVPR 2015 Yao Qin, Huchuan Lu, Yiqun Xu, He Wang

In this paper, we introduce Cellular Automata--a dynamic evolution model to intuitively detect the salient object.

Saliency Detection

Cannot find the paper you are looking for? You can Submit a new open access paper.