Search Results for author: Hao Zhu

Found 111 papers, 48 papers with code

STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians

no code implementations22 Mar 2024 Yifei Zeng, Yanqin Jiang, Siyu Zhu, Yuanxun Lu, Youtian Lin, Hao Zhu, Weiming Hu, Xun Cao, Yao Yao

Recent progress in pre-trained diffusion models and 3D generation have spurred interest in 4D content creation.

Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance

no code implementations21 Mar 2024 Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, Siyu Zhu

In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion guidance in curernt human generative techniques.

Image Animation

SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents

1 code implementation13 Mar 2024 Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Graham Neubig, Yonatan Bisk, Hao Zhu

Motivated by this gap, we propose an interactive learning method, SOTOPIA-$\pi$, improving the social intelligence of language agents.

Language Modelling Large Language Model

Timed-Elastic-Band Based Variable Splitting for Autonomous Trajectory Planning

no code implementations5 Feb 2024 Hao Zhu, Kefan Jin, Rui Gao, Jialin Wang, C. -J. Richard Shi

Existing trajectory planning methods are struggling to handle the issue of autonomous track swinging during navigation, resulting in significant errors when reaching the destination.

Collision Avoidance Trajectory Planning

DevEval: Evaluating Code Generation in Practical Software Projects

no code implementations12 Jan 2024 Jia Li, Ge Li, YunFei Zhao, Yongmin Li, Zhi Jin, Hao Zhu, Huanyu Liu, Kaibo Liu, Lecheng Wang, Zheng Fang, Lanshen Wang, Jiazheng Ding, Xuanming Zhang, Yihong Dong, Yuqi Zhu, Bin Gu, Mengfei Yang

Compared to previous benchmarks, DevEval aligns to practical projects in multiple dimensions, e. g., real program distributions, sufficient dependencies, and enough-scale project contexts.

Code Generation

FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions

no code implementations5 Dec 2023 Zhen Liu, Hao Zhu, Qi Zhang, Jingde Fu, Weibing Deng, Zhan Ma, Yanwen Guo, Xun Cao

Implicit Neural Representation (INR), which utilizes a neural network to map coordinate inputs to corresponding attributes, is causing a revolution in the field of signal processing.

VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Prior

no code implementations4 Dec 2023 Xusen Sun, Longhao Zhang, Hao Zhu, Peng Zhang, Bang Zhang, Xinya Ji, Kangneng Zhou, Daiheng Gao, Liefeng Bo, Xun Cao

Audio-driven talking head generation has drawn much attention in recent years, and many efforts have been made in lip-sync, expressive facial expressions, natural head pose generation, and high video quality.

Talking Head Generation

Relightable 3D Gaussian: Real-time Point Cloud Relighting with BRDF Decomposition and Ray Tracing

no code implementations27 Nov 2023 Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, Yao Yao

We present a novel differentiable point-based rendering framework for material and lighting decomposition from multi-view images, enabling editing, ray-tracing, and real-time relighting of the 3D point cloud.

BRDF estimation Lighting Estimation

Multi-intention Inverse Q-learning for Interpretable Behavior Representation

no code implementations23 Nov 2023 Hao Zhu, Brice De La Crompe, Gabriel Kalweit, Artur Schneider, Maria Kalweit, Ilka Diester, Joschka Boedecker

In advancing the understanding of decision-making processes, Inverse Reinforcement Learning (IRL) have proven instrumental in reconstructing animal's multiple intentions amidst complex behaviors.

Decision Making Q-Learning

Alignment and Outer Shell Isotropy for Hyperbolic Graph Contrastive Learning

no code implementations27 Oct 2023 Yifei Zhang, Hao Zhu, Jiahong Liu, Piotr Koniusz, Irwin King

We show that in the hyperbolic space one has to address the leaf- and height-level uniformity which are related to properties of trees, whereas in the ambient space of the hyperbolic manifold, these notions translate into imposing an isotropic ring density towards boundaries of Poincar\'e ball.

Contrastive Learning Graph Embedding +1

SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents

1 code implementation18 Oct 2023 Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and evaluate their social intelligence.

PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation

no code implementations14 Oct 2023 Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Weidong Cai, Wayne Wu

We first propose a novel score function, Denoised Score Distillation (DSD), which directly modifies the SDS by introducing negative gradient components to iteratively correct the gradient direction and generate high-quality textures.

Text to 3D text-to-3d-human +1

Tracking Anything in Heart All at Once

no code implementations4 Oct 2023 Chengkang Shen, Hao Zhu, You Zhou, Yu Liu, Si Yi, Lili Dong, Weipeng Zhao, David J. Brady, Xun Cao, Zhan Ma, Yi Lin

Myocardial motion tracking stands as an essential clinical tool in the prevention and detection of Cardiovascular Diseases (CVDs), the foremost cause of death globally.

Motion Estimation

Data-driven Forced Oscillation Localization using Inferred Impulse Responses

1 code implementation2 Oct 2023 Shaohui Liu, Hao Zhu, Vassilis Kekatos

Poorly damped oscillations pose threats to the stability and reliability of interconnected power systems.

RHINO: Regularizing the Hash-based Implicit Neural Representation

no code implementations22 Sep 2023 Hao Zhu, Fengyi Liu, Qi Zhang, Xun Cao, Zhan Ma

This connection ensures a seamless backpropagation of gradients from the network's output back to the input coordinates, thereby enhancing regularization.

Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail

no code implementations19 Sep 2023 Yiyu Zhuang, Qi Zhang, Ying Feng, Hao Zhu, Yao Yao, Xiaoyu Li, Yan-Pei Cao, Ying Shan, Xun Cao

Drawing inspiration from voxel-based representations with the level of detail (LoD), we introduce a multi-scale tri-plane-based scene representation that is capable of capturing the LoD of the signed distance function (SDF) and the space radiance.

Surface Reconstruction

WebArena: A Realistic Web Environment for Building Autonomous Agents

1 code implementation25 Jul 2023 Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

Building upon our environment, we release a set of benchmark tasks focusing on evaluating the functional correctness of task completions.

Topology-aware Piecewise Linearization of the AC Power Flow through Generative Modeling

no code implementations24 Jul 2023 Young-ho Cho, Hao Zhu

Effective power flow modeling critically affects the ability to efficiently solve large-scale grid optimization problems, especially those with topology-related decision variables.

Interpretable Neural Embeddings with Sparse Self-Representation

no code implementations25 Jun 2023 Minxue Xia, Hao Zhu

Many methods employ sparse representation to learn interpretable word embeddings for better interpretability.

Dictionary Learning Word Embeddings

Cascade Subspace Clustering for Outlier Detection

no code implementations23 Jun 2023 Qi Yang, Hao Zhu

A suitable Markov Chain can be defined on the self-representation and it allows us to recognize the difference between inliers and outliers.

Clustering Outlier Detection

AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation

no code implementations16 Jun 2023 Yifei Zeng, Yuanxun Lu, Xinya Ji, Yao Yao, Hao Zhu, Xun Cao

Unlike previous approaches that can only synthesize avatars based on simple text descriptions, our method enables the creation of personalized avatars from casually captured face or body images, while still supporting text-based model generation and editing.

Text to 3D

COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements

no code implementations3 Jun 2023 Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, Maarten Sap

To study the contextual dynamics of offensiveness, we train models to generate COBRA explanations, with and without access to the context.

Hierarchical Prompting Assists Large Language Model on Web Navigation

3 code implementations23 May 2023 Abishek Sridhar, Robert Lo, Frank F. Xu, Hao Zhu, Shuyan Zhou

Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks.

Decision Making Language Modelling +1

High-Fidelity 3D Face Generation from Natural Language Descriptions

1 code implementation CVPR 2023 Menghua Wu, Hao Zhu, Linjia Huang, Yiyu Zhuang, Yuanxun Lu, Xun Cao

Synthesizing high-quality 3D face models from natural language descriptions is very valuable for many applications, including avatar creation, virtual reality, and telepresence.

Descriptive Face Generation +4

NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination

no code implementations18 Apr 2023 Yiyu Zhuang, Qi Zhang, Xuan Wang, Hao Zhu, Ying Feng, Xiaoyu Li, Ying Shan, Xun Cao

Recent advances in implicit neural representation have demonstrated the ability to recover detailed geometry and material from multi-view images.

From Saliency to DINO: Saliency-guided Vision Transformer for Few-shot Keypoint Detection

no code implementations6 Apr 2023 Changsheng Lu, Hao Zhu, Piotr Koniusz

Unlike current deep keypoint detectors that are trained to recognize limited number of body parts, few-shot keypoint detection (FSKD) attempts to localize any keypoints, including novel or base keypoints, depending on the reference samples.

Keypoint Detection

Disorder-invariant Implicit Neural Representation

no code implementations3 Apr 2023 Hao Zhu, Shaowen Xie, Zhen Liu, Fengyi Liu, Qi Zhang, You Zhou, Yi Lin, Zhan Ma, Xun Cao

However, the expressive power of INR is limited by the spectral bias in the network training.

Attribute Retrieval

CelebV-Text: A Large-Scale Facial Text-Video Dataset

1 code implementation CVPR 2023 Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Weidong Cai, Wayne Wu

This paper presents CelebV-Text, a large-scale, diverse, and high-quality dataset of facial text-video pairs, to facilitate research on facial text-to-video generation tasks.

Text Generation Text-to-Video Generation +1

Optimal Power System Topology Control Under Uncertain Wildfire Risk

no code implementations14 Mar 2023 Yuqi Zhou, Kaarthik Sundar, Deepjyoti Deka, Hao Zhu

Wildfires pose a significant threat to the safe and reliable operation of electric power systems.

Computational Language Acquisition with Theory of Mind

1 code implementation2 Mar 2023 Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk, Graham Neubig

We also find some evidence that increasing task difficulty in the training process results in more fluent and precise utterances in evaluation.

Language Acquisition

EXCALIBUR: Encouraging and Evaluating Embodied Exploration

no code implementations CVPR 2023 Hao Zhu, Raghav Kapoor, So Yeon Min, Winson Han, Jiatai Li, Kaiwen Geng, Graham Neubig, Yonatan Bisk, Aniruddha Kembhavi, Luca Weihs

Humans constantly explore and learn about their environment out of curiosity, gather information, and update their models of the world.

Wind Power Scenario Generation Using Graph Convolutional Generative Adversarial Network

no code implementations19 Dec 2022 Young-ho Cho, Shaohui Liu, Duehee Lee, Hao Zhu

Generating wind power scenarios is very important for studying the impacts of multiple wind farms that are interconnected to the grid.

Generative Adversarial Network

Spectral Feature Augmentation for Graph Contrastive Learning and Beyond

no code implementations2 Dec 2022 Yifei Zhang, Hao Zhu, Zixing Song, Piotr Koniusz, Irwin King

Although augmentations (e. g., perturbation of graph edges, image crops) boost the efficiency of Contrastive Learning (CL), feature level augmentation is another plausible, complementary yet not well researched strategy.

Contrastive Learning

DINER: Disorder-Invariant Implicit Neural Representation

no code implementations CVPR 2023 Shaowen Xie, Hao Zhu, Zhen Liu, Qi Zhang, You Zhou, Xun Cao, Zhan Ma

Implicit neural representation (INR) characterizes the attributes of a signal as a function of corresponding coordinates which emerges as a sharp weapon for solving inverse problems.

Retrieval

Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue

1 code implementation10 Oct 2022 So Yeon Min, Hao Zhu, Ruslan Salakhutdinov, Yonatan Bisk

We provide empirical comparisons of metrics, analysis of three models, and make suggestions for how the field might best progress.

Imitation Learning Instruction Following

Dynamic Response Recovery Using Ambient Synchrophasor Data: A Synthetic Texas Interconnection Case Study

1 code implementation22 Sep 2022 Shaohui Liu, Hao Zhu, Vassilis Kekatos

Wide-area dynamic studies are of paramount importance to ensure the stability and reliability of power grids.

CelebV-HQ: A Large-Scale Video Facial Attributes Dataset

1 code implementation25 Jul 2022 Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, Chen Change Loy

Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.

Attribute Face Generation +1

An Unsupervised Deep-Learning Method for Bone Age Assessment

no code implementations12 Jun 2022 Hao Zhu, Wan-Jing Nie, Yue-Jie Hou, Qi-Meng Du, Si-Jing Li, Chi-Chun Zhou

In this paper, based on the convolutional auto-encoder with constraints (CCAE), an unsupervised deep-learning model proposed in the classification of the fingerprint, we propose this model for the classification of the bone age and baptize it BA-CCAE.

COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

1 code implementation9 Jun 2022 Yifei Zhang, Hao Zhu, Zixing Song, Piotr Koniusz, Irwin King

In this paper, we show that the node embedding obtained via the graph augmentations is highly biased, somewhat limiting contrastive models from learning discriminative features for downstream tasks.

Contrastive Learning Graph Representation Learning

Topology-aware Graph Neural Networks for Learning Feasible and Adaptive ac-OPF Solutions

1 code implementation16 May 2022 Shaohui Liu, Chengyang Wu, Hao Zhu

We develop a new topology-informed graph neural network (GNN) approach for predicting the optimal solutions of real-time ac-OPF problem.

Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

1 code implementation13 Apr 2022 Haiming Yu, Hao Zhu, Xiangju Lu, Junhui Liu

In this work, we propose MobileFSGAN, a novel lightweight GAN for face swap that can run on mobile devices with much fewer parameters while achieving competitive performance.

Attribute Image Generation

Epipolar Focus Spectrum: A Novel Light Field Representation and Application in Dense-view Reconstruction

no code implementations1 Apr 2022 Yaning Li, Xue Wang, Hao Zhu, Guoqing Zhou, Qing Wang

Existing light field representations, such as epipolar plane image (EPI) and sub-aperture images, do not consider the structural characteristics across the views, so they usually require additional disparity and spatial structure cues for follow-up tasks.

Ensemble Spectral Prediction (ESP) Model for Metabolite Annotation

no code implementations25 Mar 2022 Xinmeng Li, Hao Zhu, Li-Ping Liu, Soha Hassoun

We show that annotation performance, for ESP and other models, is a strong function of the number of molecules in the candidate set and their similarity to the target molecule.

SSGCNet: A Sparse Spectra Graph Convolutional Network for Epileptic EEG Signal Classification

no code implementations24 Mar 2022 Jialin Wang, Rui Gao, Haotian Zheng, Hao Zhu, C. -J. Richard Shi

Compared with the existing literature, our WNFG of EEG signals achieves up to 10 times of redundant edge reduction, and our approach achieves up to 97 times of model pruning without loss of classification accuracy.

Classification EEG +1

Graph-adaptive Rectified Linear Unit for Graph Neural Networks

no code implementations13 Feb 2022 Yifei Zhang, Hao Zhu, Ziqiao Meng, Piotr Koniusz, Irwin King

However, in the updating stage, all nodes share the same updating function.

Contrastive Laplacian Eigenmaps

1 code implementation NeurIPS 2021 Hao Zhu, Ke Sun, Piotr Koniusz

Starting from a GAN-inspired contrastive formulation, we show that the Jensen-Shannon divergence underlying many contrastive graph embedding models fails under disjoint positive and negative distributions, which may naturally emerge during sampling in the contrastive setting.

Contrastive Learning Graph Embedding

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

1 code implementation4 Jan 2022 Yunze Xiao, Hao Zhu, Haotian Yang, Zhengyu Diao, Xiangju Lu, Xun Cao

By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape.

EASE: Unsupervised Discriminant Subspace Learning for Transductive Few-Shot Learning

1 code implementation CVPR 2022 Hao Zhu, Piotr Koniusz

We present an unsupErvised discriminAnt Subspace lEarning (EASE) that improves transductive few-shot learning performance by learning a linear projection onto a subspace built from features of the support set and the unlabeled query set in the test time.

Few-Shot Learning

MoFaNeRF: Morphable Facial Neural Radiance Field

1 code implementation4 Dec 2021 Yiyu Zhuang, Hao Zhu, Xusen Sun, Xun Cao

To the best of our knowledge, our work is the first facial parametric model built upon a neural radiance field that can be used in fitting, generation and manipulation.

Image Generation Novel View Synthesis

Appliance Level Short-term Load Forecasting via Recurrent Neural Network

no code implementations23 Nov 2021 Yuqi Zhou, Arun Sukumaran Nair, David Ganger, Abhinandan Tripathi, Chaitanya Baone, Hao Zhu

Accurate load forecasting is critical for electricity market operations and other real-time decision-making tasks in power systems.

Decision Making Load Forecasting

Scalable Learning for Optimal Load Shedding Under Power Grid Emergency Operations

no code implementations23 Nov 2021 Yuqi Zhou, Jeehyun Park, Hao Zhu

Effective and timely responses to unexpected contingencies are crucial for enhancing the resilience of power grids.

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction

1 code implementation1 Nov 2021 Hao Zhu, Haotian Yang, Longwei Guo, Yidi Zhang, Yanru Wang, Mingkai Huang, Menghua Wu, Qiu Shen, Ruigang Yang, Xun Cao

By training on FaceScape data, a novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input.

3D Face Reconstruction 3D Reconstruction

On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structure

no code implementations14 Oct 2021 Lintao Ye, Hao Zhu, Vijay Gupta

We study the problem of control policy design for decentralized state-feedback linear quadratic control with a partially nested information structure, when the system model is unknown.

Risk-Aware Learning for Scalable Voltage Optimization in Distribution Grids

1 code implementation4 Oct 2021 Shanny Lin, Shaohui Liu, Hao Zhu

Real-time coordination of distributed energy resources (DERs) is crucial for regulating the voltage profile in distribution grids.

Decision Making

Few-shot graph link prediction with domain adaptation

no code implementations29 Sep 2021 Hao Zhu, Mahashweta Das, Mangesh Bendre, Fei Wang, Hao Yang, Soha Hassoun

In this work, we propose an adversarial training based modification to the current state-of-the-arts link prediction method to solve this problem.

Domain Adaptation Few-Shot Learning +1

Dependency Induction Through the Lens of Visual Perception

1 code implementation CoNLL (EMNLP) 2021 Ruisi Su, Shruti Rijhwani, Hao Zhu, Junxian He, Xinyu Wang, Yonatan Bisk, Graham Neubig

Our experiments find that concreteness is a strong indicator for learning dependency grammars, improving the direct attachment score (DAS) by over 50\% as compared to state-of-the-art models trained on pure text.

Constituency Grammar Induction Dependency Parsing

REFINE: Random RangE FInder for Network Embedding

no code implementations24 Aug 2021 Hao Zhu, Piotr Koniusz

Moreover, we design a simple but efficient spectral filter for network enhancement to obtain higher-order information for node representation.

Network Embedding Node Classification

CIGLI: Conditional Image Generation from Language & Image

1 code implementation20 Aug 2021 Xiaopeng Lu, Lynnette Ng, Jared Fernandez, Hao Zhu

Instead of generating an image based on text as in text-image generation, this task requires the generation of an image from a textual description and an image prompt.

Conditional Image Generation

Data-driven Modeling for Distribution Grids Under Partial Observability

no code implementations18 Aug 2021 Shanny Lin, Hao Zhu

Accurately modeling power distribution grids is crucial for designing effective monitoring and decision making algorithms.

Decision Making

Efficient Representation for Electric Vehicle Charging Station Operations using Reinforcement Learning

no code implementations7 Aug 2021 Kyung-bin Kwon, Hao Zhu

Effectively operating electrical vehicle charging station (EVCS) is crucial for enabling the rapid transition of electrified transportation.

reinforcement-learning Reinforcement Learning (RL)

Detailed Avatar Recovery from Single Image

no code implementations6 Aug 2021 Hao Zhu, Xinxin Zuo, Haotian Yang, Sen Wang, Xun Cao, Ruigang Yang

In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation.

Graph Convolutional Network with Generalized Factorized Bilinear Aggregation

no code implementations24 Jul 2021 Hao Zhu, Piotr Koniusz

Although Graph Convolutional Networks (GCNs) have demonstrated their power in various applications, the graph convolutional layers, as the most important component of GCN, are still using linear transformations and a simple pooling step.

text-classification Text Classification

Few-shot Language Coordination by Modeling Theory of Mind

no code implementations12 Jul 2021 Hao Zhu, Graham Neubig, Yonatan Bisk

Positive results from our experiments hint at the importance of explicitly modeling communication as a socio-pragmatic progress.

Graph Neural Networks for Learning Real-Time Prices in Electricity Market

1 code implementation19 Jun 2021 Shaohui Liu, Chengyang Wu, Hao Zhu

Solving the optimal power flow (OPF) problem in real-time electricity market improves the efficiency and reliability in the integration of low-carbon energy resources into the power grids.

Inferring power system dynamics from synchrophasor data using Gaussian processes

no code implementations26 May 2021 Mana Jalali, Vassilis Kekatos, Siddharth Bhela, Hao Zhu, Virgilio Centeno

Synchrophasor data provide unprecedented opportunities for inferring power system dynamics, such as estimating voltage angles, frequencies, and accelerations along with power injection at all buses.

Gaussian Processes Uncertainty Quantification

A Dynamic Response Recovery Framework Using Ambient Synchrophasor Data

1 code implementation12 Apr 2021 Shaohui Liu, Hao Zhu, Vassilis Kekatos

Wide-area dynamic studies are of paramount importance to ensure the stability and reliability of power grids.

Enhancing the Spatio-temporal Observability of Grid-Edge Resources in Distribution Grids

no code implementations15 Feb 2021 Shanny Lin, Hao Zhu

The load recovery solutions can be utilized to identify the EV charging events at each load node and to infer the total behind-the-meter PV output.

Deep Anti-aliasing of Whole Focal Stack Using Slice Spectrum

no code implementations23 Jan 2021 Yaning Li, Xue Wang, Hao Zhu, Guoqing Zhou, Qing Wang

Based on these two observations, we propose a learning-based FSS reconstruction approach for one-time aliasing removing over the whole focal stack.

Depth Estimation

Simple Spectral Graph Convolution

2 code implementations ICLR 2021 Hao Zhu, Piotr Koniusz

Our spectral analysis shows that our simple spectral graph convolution used in S^2GC is a low-pass filter which partitions networks into a few large parts.

Clustering Node Classification +2

Imbalance Robust Softmax for Deep Embeeding Learning

no code implementations23 Nov 2020 Hao Zhu, Yang Yuan, Guosheng Hu, Xiang Wu, Neil Robertson

IR-Softmax can generalise to any softmax and its variants (which are discriminative for open-set problem) by directly setting the weights as their class centers, naturally solving the data imbalance problem.

Face Recognition Person Re-Identification

Lets Play Music: Audio-driven Performance Video Generation

no code implementations5 Nov 2020 Hao Zhu, Yi Li, Feixia Zhu, Aihua Zheng, Ran He

We propose a new task named Audio-driven Per-formance Video Generation (APVG), which aims to synthesizethe video of a person playing a certain instrument guided bya given music audio clip.

Video Generation

AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection

no code implementations NeurIPS 2020 Hao Zhu, Chaoyou Fu, Qianyi Wu, Wayne Wu, Chen Qian, Ran He

However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation.

Multi-agent Trajectory Prediction with Fuzzy Query Attention

1 code implementation NeurIPS 2020 Nitin Kamra, Hao Zhu, Dweep Trivedi, Ming Zhang, Yan Liu

Trajectory prediction for scenes with multiple agents and entities is a challenging problem in numerous domains such as traffic prediction, pedestrian tracking and path planning.

Decision Making Traffic Prediction +1

Using Graph Neural Networks for Mass Spectrometry Prediction

no code implementations9 Oct 2020 Hao Zhu, LiPing Liu, Soha Hassoun

We compare our results to NEIMS, a neural network model that utilizes molecular fingerprints as inputs.

The Return of Lexical Dependencies: Neural Lexicalized PCFGs

3 code implementations29 Jul 2020 Hao Zhu, Yonatan Bisk, Graham Neubig

In this paper we demonstrate that $\textit{context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies}$.

Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses

1 code implementation17 Jul 2020 Miao Liao, Sibo Zhang, Peng Wang, Hao Zhu, Xinxin Zuo, Ruigang Yang

In this paper, we propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics.

Generative Adversarial Network

Additively Homomorphical Encryption based Deep Neural Network for Asymmetrically Collaborative Machine Learning

no code implementations14 Jul 2020 Yifei Zhang, Hao Zhu

For this scheme, we propose a novel privacy-preserving architecture where two parties can collaboratively train a deep learning model efficiently while preserving the privacy of each party's data.

BIG-bench Machine Learning Privacy Preserving

FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction

1 code implementation CVPR 2020 Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, Xun Cao

In this paper, we present a large-scale detailed 3D face dataset, FaceScape, and propose a novel algorithm that is able to predict elaborate riggable 3D face models from a single image input.

SAPIEN: A SimulAted Part-based Interactive ENvironment

1 code implementation CVPR 2020 Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su

To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.

Attribute

Deep Audio-Visual Learning: A Survey

no code implementations14 Jan 2020 Hao Zhu, Mandi Luo, Rui Wang, Aihua Zheng, Ran He

Audio-visual learning, aimed at exploiting the relationship between audio and visual modalities, has drawn considerable attention since deep learning started to be used successfully.

audio-visual learning Representation Learning

EnsemFDet: An Ensemble Approach to Fraud Detection based on Bipartite Graph

no code implementations23 Dec 2019 Yuxiang Ren, Hao Zhu, Jiawei Zhang, Peng Dai, Liefeng Bo

Existing fraud detection methods try to identify unexpected dense subgraphs and treat related nodes as suspicious.

Fraud Detection

S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenes

1 code implementation31 Oct 2019 Yuzhe Qin, Rui Chen, Hao Zhu, Meng Song, Jing Xu, Hao Su

Grasping is among the most fundamental and long-lasting problems in robotics study.

FewRel 2.0: Towards More Challenging Few-Shot Relation Classification

1 code implementation IJCNLP 2019 Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie zhou

We present FewRel 2. 0, a more challenging task to investigate two aspects of few-shot relation classification models: (1) Can they adapt to a new domain with only a handful of instances?

Classification Domain Adaptation +3

Learned Point Cloud Geometry Compression

2 code implementations26 Sep 2019 Jianqiang Wang, Hao Zhu, Zhan Ma, Tong Chen, Haojie Liu, Qiu Shen

This paper presents a novel end-to-end Learned Point Cloud Geometry Compression (a. k. a., Learned-PCGC) framework, to efficiently compress the point cloud geometry (PCG) using deep neural networks (DNN) based variational autoencoders (VAE).

Surface Reconstruction

Quantifying Similarity between Relations with Fact Distribution

1 code implementation ACL 2019 Weize Chen, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun

We introduce a conceptually simple and effective method to quantify the similarity between relations in knowledge bases.

General Classification Open Information Extraction

Ekar: An Explainable Method for Knowledge Aware Recommendation

2 code implementations22 Jun 2019 Weiping Song, Zhijian Duan, Ziqing Yang, Hao Zhu, Ming Zhang, Jian Tang

Recently, a variety of methods have been developed for this problem, which generally try to learn effective representations of users and items and then match items to users according to their representations.

Knowledge-Aware Recommendation Knowledge Graphs +1

Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies

no code implementations19 Jun 2019 Kaiqing Zhang, Alec Koppel, Hao Zhu, Tamer Başar

Under a further strict saddle points assumption, this result establishes convergence to essentially locally-optimal policies of the underlying problem, and thus bridges the gap in existing literature on the convergence of PG methods.

Autonomous Driving Policy Gradient Methods

Doc2hash: Learning Discrete Latent variables for Documents Retrieval

1 code implementation NAACL 2019 Yifei Zhang, Hao Zhu

However, the discrete stochastic layer is usually incompatible with the backpropagation in the training stage, and thus causes a gradient flow problem because of non-differentiable operators.

Retrieval

Neural Finite-State Transducers: Beyond Rational Relations

no code implementations NAACL 2019 Chu-Cheng Lin, Hao Zhu, Matthew R. Gormley, Jason Eisner

We introduce neural finite state transducers (NFSTs), a family of string transduction models defining joint and conditional probability distributions over pairs of strings.

Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation

1 code implementation CVPR 2019 Hao Zhu, Xinxin Zuo, Sen Wang, Xun Cao, Ruigang Yang

This paper presents a novel framework to recover detailed human body shapes from a single image.

Breaking the Spatio-Angular Trade-off for Light Field Super-Resolution via LSTM Modelling on Epipolar Plane Images

no code implementations15 Feb 2019 Hao Zhu, Mantang Guo, Hongdong Li, Qing Wang, Antonio Robles-Kelly

We prove that the light field is a 2D series, thus, a specifically designed CNN-LSTM network is proposed to capture the continuity property of the EPI.

Super-Resolution

Arbitrary Talking Face Generation via Attentional Audio-Visual Coherence Learning

no code implementations17 Dec 2018 Hao Zhu, Huaibo Huang, Yi Li, Aihua Zheng, Ran He

Talking face generation aims to synthesize a face video with precise lip synchronization as well as a smooth transition of facial motion over the entire video via the given speech clip and facial image.

Talking Face Generation

CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark

3 code implementations CVPR 2019 Jiefeng Li, Can Wang, Hao Zhu, Yihuan Mao, Hao-Shu Fang, Cewu Lu

In this paper, we propose a novel and efficient method to tackle the problem of pose estimation in the crowd and a new dataset to better evaluate algorithms.

Keypoint Detection Multi-Person Pose Estimation

Language Modeling with Sparse Product of Sememe Experts

1 code implementation EMNLP 2018 Yihong Gu, Jun Yan, Hao Zhu, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin

Most language modeling methods rely on large-scale data to statistically learn the sequential patterns of words.

Language Modelling

Cross-lingual Lexical Sememe Prediction

1 code implementation EMNLP 2018 Fanchao Qi, Yankai Lin, Maosong Sun, Hao Zhu, Ruobing Xie, Zhiyuan Liu

We propose a novel framework to model correlations between sememes and multi-lingual words in low-dimensional semantic space for sememe prediction.

Learning Word Embeddings Multilingual Word Embeddings

Incorporating Chinese Characters of Words for Lexical Sememe Prediction

1 code implementation ACL 2018 Huiming Jin, Hao Zhu, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin

However, existing methods of lexical sememe prediction typically rely on the external context of words to represent the meaning, which usually fails to deal with low-frequency and out-of-vocabulary words.

Common Sense Reasoning

Barotropic instability of shear flows

no code implementations3 Jan 2018 Zhiwu Lin, Jincheng Yang, Hao Zhu

The second one is to write the linearized fluid equation in a Hamiltonian form and then use an instability index theory for general Hamiltonian PDEs.

Analysis of PDEs

Light Field Segmentation From Super-pixel Graph Representation

no code implementations20 Dec 2017 Xianqiang Lv, Hao Zhu, Qing Wang

The large volume of input data and the redundancy of light field make it an open challenge.

Segmentation

4D Light Field Superpixel and Segmentation

no code implementations CVPR 2017 Hao Zhu, Qi Zhang, Qing Wang

Superpixel segmentation of 2D image has been widely used in many computer vision tasks.

Segmentation

Decentralized learning for wireless communications and networking

no code implementations30 Mar 2015 Georgios B. Giannakis, Qing Ling, Gonzalo Mateos, Ioannis D. Schizas, Hao Zhu

This chapter deals with decentralized learning algorithms for in-network processing of graph-valued data.

Spectrum Cartography

Cannot find the paper you are looking for? You can Submit a new open access paper.