Search Results for author: Agustinus Kristiadi

Found 27 papers, 15 papers with code

Towards Cost-Effective Reward Guided Text Generation

no code implementations6 Feb 2025 Ahmad Rashid, Ruotian Wu, Rongqi Fan, Hongliang Li, Agustinus Kristiadi, Pascal Poupart

However, they rely on a reward model to score each candidate token generated by the language model at inference, incurring significant test-time overhead.

Text Generation

Efficient Weight-Space Laplace-Gaussian Filtering and Smoothing for Sequential Deep Learning

no code implementations9 Oct 2024 Joanna Sliwa, Frank Schneider, Nathanael Bosch, Agustinus Kristiadi, Philipp Hennig

Efficiently learning a sequence of related tasks, such as in continual learning, poses a significant challenge for neural nets due to the delicate trade-off between catastrophic forgetting and loss of plasticity.

Bayesian Inference Continual Learning

Uncertainty-Guided Optimization on Large Language Model Search Trees

1 code implementation4 Jul 2024 Julia Grosse, Ruotian Wu, Ahmad Rashid, Philipp Hennig, Pascal Poupart, Agustinus Kristiadi

These beliefs are useful for defining a sample-based, non-myopic acquisition function that allows for a more data-efficient exploration scheme than standard search algorithms on LLMs.

Bayesian Optimization Efficient Exploration +3

A Critical Look At Tokenwise Reward-Guided Text Generation

no code implementations12 Jun 2024 Ahmad Rashid, Ruotian Wu, Julia Grosse, Agustinus Kristiadi, Pascal Poupart

Due to their ability to bypass LLM finetuning, tokenwise reward-guided text generation (RGTG) methods have recently been proposed.

Text Generation

How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization?

1 code implementation10 Jun 2024 Agustinus Kristiadi, Felix Strieth-Kalthoff, Sriram Ganapathi Subramanian, Vincent Fortuin, Pascal Poupart, Geoff Pleiss

The gathered feedback is used to learn a Bayesian preference model that can readily be incorporated into the BO thread, to steer its exploration-exploitation process.

Bayesian Optimization Blocking +1

A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?

1 code implementation7 Feb 2024 Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta, Pascal Poupart, Alán Aspuru-Guzik, Geoff Pleiss

Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior domain knowledge into efficient exploration of a large molecular space.

Bayesian Optimization Efficient Exploration

Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks

1 code implementation7 Nov 2023 Ahmad Rashid, Serena Hacker, Guojun Zhang, Agustinus Kristiadi, Pascal Poupart

For instance, ReLU networks - a popular class of neural network architectures - have been shown to almost always yield high confidence predictions when the test data are far away from the training set, even when they are trained with OOD data.

On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective

no code implementations29 Sep 2023 Jonathan Wenger, Felix Dangel, Agustinus Kristiadi

Kernel methods are theoretically well-understood and as a result enjoy algorithmic benefits, which can be demonstrated to hold in wide synthetic neural network architectures.

Continual Learning Uncertainty Quantification

Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization

1 code implementation17 Apr 2023 Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Vincent Fortuin

The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks.

Bayesian Optimization Decision Making +3

Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks

1 code implementation20 May 2022 Agustinus Kristiadi, Runa Eschenhagen, Philipp Hennig

We show that the resulting posterior approximation is competitive with even the gold-standard full-batch Hamiltonian Monte Carlo.

Laplace Redux -- Effortless Bayesian Deep Learning

6 code implementations NeurIPS 2021 Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, Philipp Hennig

Bayesian formulations of deep learning have been shown to have compelling theoretical properties and offer practical functional benefits, such as improved predictive uncertainty quantification and model selection.

Deep Learning Misconceptions +2

Being a Bit Frequentist Improves Bayesian Neural Networks

1 code implementation18 Jun 2021 Agustinus Kristiadi, Matthias Hein, Philipp Hennig

Despite their compelling theoretical properties, Bayesian neural networks (BNNs) tend to perform worse than frequentist methods in classification-based uncertainty quantification (UQ) tasks such as out-of-distribution (OOD) detection.

Bayesian Inference Out of Distribution (OOD) Detection +1

Laplace Redux - Effortless Bayesian Deep Learning

no code implementations NeurIPS 2021 Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, Philipp Hennig

Bayesian formulations of deep learning have been shown to have compelling theoretical properties and offer practical functional benefits, such as improved predictive uncertainty quantification and model selection.

Deep Learning Misconceptions +2

Learnable Uncertainty under Laplace Approximations

1 code implementation6 Oct 2020 Agustinus Kristiadi, Matthias Hein, Philipp Hennig

Laplace approximations are classic, computationally lightweight means for constructing Bayesian neural networks (BNNs).

Uncertainty Quantification

An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence

no code implementations NeurIPS 2021 Agustinus Kristiadi, Matthias Hein, Philipp Hennig

We extend finite ReLU BNNs with infinite ReLU features via the GP and show that the resulting model is asymptotically maximally uncertain far away from the data while the BNNs' predictive power is unaffected near the data.

Multi-class Classification

Fixing Asymptotic Uncertainty of Bayesian Neural Networks with Infinite ReLU Features

no code implementations28 Sep 2020 Agustinus Kristiadi, Matthias Hein, Philipp Hennig

However, far away from the training data, even Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be overconfident.

Multi-class Classification

Fast Predictive Uncertainty for Classification with Bayesian Deep Networks

1 code implementation2 Mar 2020 Marius Hobbhahn, Agustinus Kristiadi, Philipp Hennig

We reconsider old work (Laplace Bridge) to construct a Dirichlet approximation of this softmax output distribution, which yields an analytic map between Gaussian distributions in logit space and Dirichlet distributions (the conjugate prior to the Categorical distribution) in the output space.

Classification General Classification

Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks

1 code implementation ICML 2020 Agustinus Kristiadi, Matthias Hein, Philipp Hennig

These theoretical results validate the usage of last-layer Bayesian approximation and motivate a range of a fidelity-cost trade-off.

Bayesian Inference

Compound Density Networks

no code implementations ICLR 2019 Agustinus Kristiadi, Asja Fischer

Despite the huge success of deep neural networks (NNs), finding good mechanisms for quantifying their prediction uncertainty is still an open problem.

scoring rule

Predictive Uncertainty Quantification with Compound Density Networks

no code implementations4 Feb 2019 Agustinus Kristiadi, Sina Däubener, Asja Fischer

Despite the huge success of deep neural networks (NNs), finding good mechanisms for quantifying their prediction uncertainty is still an open problem.

Bayesian Inference Uncertainty Quantification

Incorporating Literals into Knowledge Graph Embeddings

1 code implementation3 Feb 2018 Agustinus Kristiadi, Mohammad Asif Khan, Denis Lukovnikov, Jens Lehmann, Asja Fischer

Most of the existing work on embedding (or latent feature) based knowledge graph analysis focuses mainly on the relations between entities.

Entity Embeddings Knowledge Graph Embeddings +2

Cannot find the paper you are looking for? You can Submit a new open access paper.