Search Results for author: Jiawei Huang

Found 33 papers, 11 papers with code

Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL

no code implementations8 Feb 2024 Jiawei Huang, Niao He, Andreas Krause

We study the sample complexity of reinforcement learning (RL) in Mean-Field Games (MFGs) with model-based function approximation that requires strategic exploration to find a Nash Equilibrium policy.

Computational Efficiency Reinforcement Learning (RL)

Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis

1 code implementation16 Jan 2024 Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, Chen Zhang, Xiang Yin, Zejun Ma, Zhou Zhao

One-shot 3D talking portrait generation aims to reconstruct a 3D avatar from an unseen image, and then animate it with a reference video or audio to generate a talking portrait video.

3D Reconstruction Super-Resolution +1

On Robust Wasserstein Barycenter: The Model and Algorithm

no code implementations25 Dec 2023 Xu Wang, Jiawei Huang, Qingyuan Yang, Jinpeng Zhang

Firstly, we improve efficiency through model reducing; we reduce RWB as an augmented Wasserstein barycenter problem, which works for both fixed-RWB and free-RWB.

Computational Efficiency Data Compression

Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications

1 code implementation29 Nov 2023 Junyi Ma, Xieyuanli Chen, Jiawei Huang, Jingyi Xu, Zhen Luo, Jintao Xu, Weihao Gu, Rui Ai, Hesheng Wang

Furthermore, the standardized evaluation protocol for preset multiple tasks is also provided to compare the performance of all the proposed baselines on present and future occupancy estimation with respect to objects of interest in autonomous driving scenarios.

Autonomous Driving

Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation

no code implementations29 May 2023 Jiawei Huang, Yi Ren, Rongjie Huang, Dongchao Yang, Zhenhui Ye, Chen Zhang, Jinglin Liu, Xiang Yin, Zejun Ma, Zhou Zhao

Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data.

Audio Generation Denoising +2

On the Statistical Efficiency of Mean Field Reinforcement Learning with General Function Approximation

no code implementations18 May 2023 Jiawei Huang, Batuhan Yardim, Niao He

In this paper, we study the fundamental statistical efficiency of Reinforcement Learning in Mean-Field Control (MFC) and Mean-Field Game (MFG) with general model-based function approximation.

GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation

no code implementations1 May 2023 Zhenhui Ye, Jinzheng He, Ziyue Jiang, Rongjie Huang, Jiawei Huang, Jinglin Liu, Yi Ren, Xiang Yin, Zejun Ma, Zhou Zhao

Recently, neural radiance field (NeRF) has become a popular rendering technique in this field since it could achieve high-fidelity and 3D-consistent talking face generation with a few-minute-long training video.

motion prediction Talking Face Generation

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

1 code implementation25 Apr 2023 Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, Yi Ren, Zhou Zhao, Shinji Watanabe

In this work, we propose a multi-modal AI system named AudioGPT, which complements LLMs (i. e., ChatGPT) with 1) foundation models to process complex audio information and solve numerous understanding and generation tasks; and 2) the input/output interface (ASR, TTS) to support spoken dialogue.

Company Competition Graph

no code implementations1 Apr 2023 Yanci Zhang, Yutong Lu, Haitao Mao, Jiawei Huang, Cien Zhang, Xinyi Li, Rui Dai

Based on the output from our system, we construct a knowledge graph with more than 700 nodes and 1200 edges.

Knowledge Graphs

Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians

no code implementations11 Mar 2023 Jiawei Huang, Akito Iizuka, Hajime Tanaka, Taku Komura, Yoshifumi Kitamura

The variance reduction speed of physically-based rendering is heavily affected by the adopted importance sampling technique.

Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models

1 code implementation30 Jan 2023 Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, Zhou Zhao

Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data.

Audio Generation Text-to-Video Generation +1

Coresets for Wasserstein Distributionally Robust Optimization Problems

1 code implementation9 Oct 2022 Ruomin Huang, Jiawei Huang, Wenjie Liu, Hu Ding

Though it is challenging to obtain a conventional coreset for \textsf{WDRO} due to the uncertainty issue of ambiguous data, we show that we can compute a ``dual coreset'' by using the strong duality property of \textsf{WDRO}.

Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites

no code implementations14 Jun 2022 Jingyang Zhang, Peng Xue, Ran Gu, Yuning Gu, Mianxin Liu, Yongsheng Pan, Zhiming Cui, Jiawei Huang, Lei Ma, Dinggang Shen

In clinical practice, a segmentation network is often required to continually learn on a sequential data stream from multiple sites rather than a consolidated set, due to the storage cost and privacy restriction.

Continual Learning

Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret

1 code implementation25 May 2022 Jiawei Huang, Li Zhao, Tao Qin, Wei Chen, Nan Jiang, Tie-Yan Liu

We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on their different tolerance on exploration risks and should be treated separately.

reinforcement-learning Reinforcement Learning (RL)

Kernel Extreme Learning Machine Optimized by the Sparrow Search Algorithm for Hyperspectral Image Classification

no code implementations3 Apr 2022 Zhixin Yan, Jiawei Huang, Kehua Xiang

To improve the classification performance and generalization ability of the hyperspectral image classification algorithm, this paper uses Multi-Scale Total Variation (MSTV) to extract the spectral features, local binary pattern (LBP) to extract spatial features, and feature superposition to obtain the fused features of hyperspectral images.

Classification Hyperspectral Image Classification

A Novel Sequential Coreset Method for Gradient Descent Algorithms

no code implementations5 Dec 2021 Jiawei Huang, Ruomin Huang, Wenjie Liu, Nikolaos M. Freris, Hu Ding

A wide range of optimization problems arising in machine learning can be solved by gradient descent algorithms, and a central question in this area is how to efficiently compress a large-scale dataset so as to reduce the computational complexity.

Data Compression

A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes

1 code implementation12 Nov 2021 Chengchun Shi, Masatoshi Uehara, Jiawei Huang, Nan Jiang

In this work, we first propose novel identification methods for OPE in POMDPs with latent confounders, by introducing bridge functions that link the target policy's value and the observed data distribution.

Off-policy evaluation

On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correction

no code implementations2 Jun 2021 Jiawei Huang, Nan Jiang

In this paper, we study the convergence properties of off-policy policy improvement algorithms with state-action density ratio correction under function approximation setting, where the objective function is formulated as a max-max-min optimization problem.

Is Simple Uniform Sampling Effective for Center-Based Clustering with Outliers: When and Why?

1 code implementation28 Feb 2021 Jiawei Huang, Wenjie Liu, Hu Ding

Real-world datasets often contain outliers, and the presence of outliers can make the clustering problems to be much more challenging.


WeightNet: Revisiting the Design Space of Weight Networks

2 code implementations ECCV 2020 Ningning Ma, Xiangyu Zhang, Jiawei Huang, Jian Sun

WeightNet is easy and memory-conserving to train, on the kernel space instead of the feature space.

Defending SVMs against Poisoning Attacks: the Hardness and DBSCAN Approach

no code implementations14 Jun 2020 Hu Ding, Fan Yang, Jiawei Huang

For the data sanitization defense, we link it to the intrinsic dimensionality of data; in particular, we provide a sampling theorem in doubling metrics for explaining the effectiveness of DBSCAN (as a density-based outlier removal method) for defending against poisoning attacks.

Minimax Value Interval for Off-Policy Evaluation and Policy Optimization

no code implementations NeurIPS 2020 Nan Jiang, Jiawei Huang

By slightly altering the derivation of previous methods (one from each style; Uehara et al., 2020), we unify them into a single value interval that comes with a special type of double robustness: when either the value-function or the importance-weight class is well specified, the interval is valid and its length quantifies the misspecification of the other class.

Efficient Exploration Off-policy evaluation +1

Minimax Weight and Q-Function Learning for Off-Policy Evaluation

no code implementations ICML 2020 Masatoshi Uehara, Jiawei Huang, Nan Jiang

We provide theoretical investigations into off-policy evaluation in reinforcement learning using function approximators for (marginalized) importance weights and value functions.

Off-policy evaluation

From Importance Sampling to Doubly Robust Policy Gradient

1 code implementation ICML 2020 Jiawei Huang, Nan Jiang

We show that on-policy policy gradient (PG) and its variance reduction variants can be derived by taking finite difference of function evaluations supplied by estimators from the importance sampling (IS) family for off-policy evaluation (OPE).

Off-policy evaluation

The Effectiveness of Uniform Sampling for Center-Based Clustering with Outliers

no code implementations24 May 2019 Hu Ding, Jiawei Huang, Haikuo Yu

The experiments suggest that the uniform sampling method can achieve comparable clustering results with other existing methods, but greatly reduce the running times.


Learning to Cluster for Proposal-Free Instance Segmentation

1 code implementation17 Mar 2018 Yen-Chang Hsu, Zheng Xu, Zsolt Kira, Jiawei Huang

We utilize the most fundamental property of instance labeling -- the pairwise relationship between pixels -- as the supervision to formulate the learning objective, then apply it to train a fully convolutional network (FCN) for learning to perform pixel-wise clustering.

Autonomous Driving Clustering +6

Robust Lane Tracking with Multi-mode Observation Model and Particle Filtering

no code implementations28 Jun 2017 Jiawei Huang, Zhaowen Wang

Automatic lane tracking involves estimating the underlying signal from a sequence of noisy signal observations.

Towards Full Automated Drive in Urban Environments: A Demonstration in GoMentum Station, California

no code implementations2 May 2017 Akansel Cosgun, Lichao Ma, Jimmy Chiu, Jiawei Huang, Mahmut Demir, Alexandre Miranda Anon, Thang Lian, Hasan Tafish, Samir Al-Stouhi

Each year, millions of motor vehicle traffic accidents all over the world cause a large number of fatalities, injuries and significant material loss.

Cannot find the paper you are looking for? You can Submit a new open access paper.