Search Results for author: Wei Deng

Found 27 papers, 14 papers with code

An Adaptive Empirical Bayesian Method for Sparse Deep Learning

1 code implementation NeurIPS 2019 Wei Deng, Xiao Zhang, Faming Liang, Guang Lin

We propose a novel adaptive empirical Bayesian method for sparse deep learning, where the sparsity is ensured via a class of self-adaptive spike-and-slab priors.

DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving

2 code implementations17 Feb 2020 Wei Deng, Junwei Pan, Tian Zhou, Deguang Kong, Aaron Flores, Guang Lin

To address the issue of significantly increased serving delay and high memory usage for ad serving in production, this paper presents \emph{DeepLight}: a framework to accelerate the CTR predictions in three aspects: 1) accelerate the model inference via explicitly searching informative feature interactions in the shallow component; 2) prune redundant layers and parameters at intra-layer and inter-layer level in the DNN component; 3) promote the sparsity of the embedding layer to preserve the most discriminant signals.

Click-Through Rate Prediction

Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications

no code implementations29 Jun 2020 Yating Wang, Wei Deng, Lin Guang

The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network.

Sparse Learning

Non-convex Learning via Replica Exchange Stochastic Gradient MCMC

2 code implementations ICML 2020 Wei Deng, Qi Feng, Liyao Gao, Faming Liang, Guang Lin

Replica exchange Monte Carlo (reMC), also known as parallel tempering, is an important technique for accelerating the convergence of the conventional Markov Chain Monte Carlo (MCMC) algorithms.

Image Classification

Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction

1 code implementation ICLR 2021 Wei Deng, Qi Feng, Georgios Karagiannis, Guang Lin, Faming Liang

Replica exchange stochastic gradient Langevin dynamics (reSGLD) has shown promise in accelerating the convergence in non-convex learning; however, an excessively large correction for avoiding biases from noisy energy estimators has limited the potential of the acceleration.

An adaptive Hessian approximated stochastic gradient MCMC method

no code implementations3 Oct 2020 Yating Wang, Wei Deng, Guang Lin

The bias introduced by stochastic approximation is controllable and can be analyzed theoretically.

A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions

2 code implementations NeurIPS 2020 Wei Deng, Guang Lin, Faming Liang

We propose an adaptively weighted stochastic gradient Langevin dynamics algorithm (SGLD), so-called contour stochastic gradient Langevin dynamics (CSGLD), for Bayesian learning in big data statistics.

Information Directed Sampling for Sparse Linear Bandits

no code implementations NeurIPS 2021 Botao Hao, Tor Lattimore, Wei Deng

Stochastic sparse linear bandits offer a practical model for high-dimensional online decision-making problems and have a rich information-regret structure.

Decision Making

Non-reversible Parallel Tempering for Uncertainty Approximation in Deep Learning

no code implementations29 Sep 2021 Wei Deng, Qian Zhang, Qi Feng, Faming Liang, Guang Lin

Parallel tempering (PT), also known as replica exchange, is the go-to workhorse for simulations of multi-modal distributions.

On Convergence of Federated Averaging Langevin Dynamics

no code implementations9 Dec 2021 Wei Deng, Qian Zhang, Yi-An Ma, Zhao Song, Guang Lin

We develop theoretical guarantees for FA-LD for strongly log-concave distributions with non-i. i. d data and study how the injected noise and the stochastic-gradient noise, the heterogeneity of data, and the varying learning rates affect the convergence.

Uncertainty Quantification

Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Art

1 code implementation23 Dec 2021 Xiang Ling, Lingfei Wu, Jiangyu Zhang, Zhenqing Qu, Wei Deng, Xiang Chen, Yaguan Qian, Chunming Wu, Shouling Ji, Tianyue Luo, Jingzheng Wu, Yanjun Wu

Then, we conduct a comprehensive and systematic review to categorize the state-of-the-art adversarial attacks against PE malware detection, as well as corresponding defenses to increase the robustness of Windows PE malware detection.

Adversarial Attack Malware Detection +2

Interacting Contour Stochastic Gradient Langevin Dynamics

1 code implementation ICLR 2022 Wei Deng, Siqi Liang, Botao Hao, Guang Lin, Faming Liang

We propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, an embarrassingly parallel multiple-chain contour stochastic gradient Langevin dynamics (CSGLD) sampler with efficient interactions.

Non-reversible Parallel Tempering for Deep Posterior Approximation

no code implementations20 Nov 2022 Wei Deng, Qian Zhang, Qi Feng, Faming Liang, Guang Lin

Notably, in big data scenarios, we obtain an appealing communication cost $O(P\log P)$ based on the optimal window size.

Provably Convergent Schrödinger Bridge with Applications to Probabilistic Time Series Imputation

1 code implementation12 May 2023 Yu Chen, Wei Deng, Shikai Fang, Fengpei Li, Nicole Tianjiao Yang, Yikai Zhang, Kashif Rasul, Shandian Zhe, Anderson Schneider, Yuriy Nevmyvaka

We show that optimizing the transport cost improves the performance and the proposed algorithm achieves the state-of-the-art result in healthcare and environmental data while exhibiting the advantage of exploring both temporal and feature patterns in probabilistic time series imputation.

Imputation Time Series

Non-convex Bayesian Learning via Stochastic Gradient Markov Chain Monte Carlo

no code implementations30 May 2023 Wei Deng

We also present the population-chain replica exchange based on non-reversibility and obtain an optimal round-trip rate for deep learning.

Uncertainty Quantification

Short-Term Multi-Horizon Line Loss Rate Forecasting of a Distribution Network Using Attention-GCN-LSTM

no code implementations19 Dec 2023 Jie Liu, Yijia Cao, Yong Li, Yixiu Guo, Wei Deng

Accurately predicting line loss rates is vital for effective line loss management in distribution networks, especially over short-term multi-horizons ranging from one hour to one week.

Management

Reflected Schrödinger Bridge for Constrained Generative Modeling

no code implementations6 Jan 2024 Wei Deng, Yu Chen, Nicole Tianjiao Yang, Hengrong Du, Qi Feng, Ricky T. Q. Chen

Diffusion models have become the go-to method for large-scale generative models in real-world applications.

Accelerating Approximate Thompson Sampling with Underdamped Langevin Monte Carlo

1 code implementation22 Jan 2024 Haoyang Zheng, Wei Deng, Christian Moya, Guang Lin

Approximate Thompson sampling with Langevin Monte Carlo broadens its reach from Gaussian posterior sampling to encompass more general smooth posteriors.

Thompson Sampling

Widely Linear Matched Filter: A Lynchpin towards the Interpretability of Complex-valued CNNs

no code implementations30 Jan 2024 Qingchen Wang, Zhe Li, Zdenka Babic, Wei Deng, Ljubiša Stanković, Danilo P. Mandic

However, applying this paradigm to illuminate the interpretability of complex-valued CNNs meets a formidable obstacle: the extension of matched filtering to a general class of noncircular complex-valued data, referred to here as the widely linear matched filter (WLMF), has been only implicit in the literature.

Faceptor: A Generalist Model for Face Perception

3 code implementations14 Mar 2024 Lixiong Qin, Mei Wang, Xuannan Liu, Yuhang Zhang, Wei Deng, Xiaoshuai Song, Weiran Xu, Weihong Deng

This design enhances the unification of model structure while improving application efficiency in terms of storage overhead.

Age Estimation Attribute +3

Can multiple-choice questions really be useful in detecting the abilities of LLMs?

1 code implementation26 Mar 2024 Wangyue Li, Liangzhi Li, Tong Xiang, Xiao Liu, Wei Deng, Noa Garcia

Additionally, we propose two methods to quantify the consistency and confidence of LLMs' output, which can be generalized to other QA evaluation benchmarks.

Multiple-choice Question Answering

Cannot find the paper you are looking for? You can Submit a new open access paper.