Search Results for author: Shengyang Sun

Found 14 papers, 8 papers with code

Information-theoretic Online Memory Selection for Continual Learning

no code implementations ICLR 2022 Shengyang Sun, Daniele Calandriello, Huiyi Hu, Ang Li, Michalis Titsias

A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams.

Continual Learning

Understanding the Variance Collapse of SVGD in High Dimensions

no code implementations ICLR 2022 Jimmy Ba, Murat A Erdogdu, Marzyeh Ghassemi, Shengyang Sun, Taiji Suzuki, Denny Wu, Tianzong Zhang

Stein variational gradient descent (SVGD) is a deterministic inference algorithm that evolves a set of particles to fit a target distribution.

Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition

2 code implementations10 Jun 2021 Shengyang Sun, Jiaxin Shi, Andrew Gordon Wilson, Roger Grosse

We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability.

Gaussian Processes

Neural Networks as Inter-Domain Inducing Points

no code implementations pproximateinference AABI Symposium 2021 Shengyang Sun, Jiaxin Shi, Roger Baker Grosse

Equivalences between infinite neural networks and Gaussian processes have been established for explaining the functional prior and training dynamics of deep learning models.

Gaussian Processes

Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?

1 code implementation6 Nov 2020 Chaoqi Wang, Shengyang Sun, Roger Grosse

While uncertainty estimation is a well-studied topic in deep learning, most such work focuses on marginal uncertainty estimates, i. e. the predictive mean and variance at individual input locations.

Active Learning

Towards Characterizing the High-dimensional Bias of Kernel-based Particle Inference Algorithms

no code implementations pproximateinference AABI Symposium 2019 Jimmy Ba, Murat A. Erdogdu, Marzyeh Ghassemi, Taiji Suzuki, Shengyang Sun, Denny Wu, Tianzong Zhang

Particle-based inference algorithm is a promising method to efficiently generate samples for an intractable target distribution by iteratively updating a set of particles.

Functional Variational Bayesian Neural Networks

2 code implementations ICLR 2019 Shengyang Sun, Guodong Zhang, Jiaxin Shi, Roger Grosse

We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defined directly on stochastic processes, i. e. distributions over functions.

Bayesian Inference Gaussian Processes +1

Differentiable Compositional Kernel Learning for Gaussian Processes

3 code implementations ICML 2018 Shengyang Sun, Guodong Zhang, Chaoqi Wang, Wenyuan Zeng, Jiaman Li, Roger Grosse

The NKN architecture is based on the composition rules for kernels, so that each unit of the network corresponds to a valid kernel.

Gaussian Processes Time Series

A Spectral Approach to Gradient Estimation for Implicit Distributions

3 code implementations ICML 2018 Jiaxin Shi, Shengyang Sun, Jun Zhu

Recently there have been increasing interests in learning and inference with implicit distributions (i. e., distributions without tractable densities).

Variational Inference

Aggregated Momentum: Stability Through Passive Damping

2 code implementations ICLR 2019 James Lucas, Shengyang Sun, Richard Zemel, Roger Grosse

Momentum is a simple and widely used trick which allows gradient-based optimizers to pick up speed along low curvature directions.

Noisy Natural Gradient as Variational Inference

2 code implementations ICML 2018 Guodong Zhang, Shengyang Sun, David Duvenaud, Roger Grosse

Variational Bayesian neural nets combine the flexibility of deep learning with Bayesian uncertainty estimation.

Active Learning Efficient Exploration +2

ZhuSuan: A Library for Bayesian Deep Learning

1 code implementation18 Sep 2017 Jiaxin Shi, Jianfei Chen, Jun Zhu, Shengyang Sun, Yucen Luo, Yihong Gu, Yuhao Zhou

In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning.

Probabilistic Programming

Kernel Implicit Variational Inference

no code implementations ICLR 2018 Jiaxin Shi, Shengyang Sun, Jun Zhu

Recent progress in variational inference has paid much attention to the flexibility of variational posteriors.

General Classification Variational Inference

Cannot find the paper you are looking for? You can Submit a new open access paper.