Search Results for author:
Found 37 papers, 15 papers with code
Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization
The performance of sequential Monte Carlo (SMC) samplers heavily depends on the tuning of the Markov kernels used in the path proposal.
Covering Multiple Objectives with a Small Set of Solutions Using Bayesian Optimization
In this work, we introduce a novel problem setting that departs from this paradigm: finding a smaller set of K solutions, where K < T, that collectively "covers" the T objectives.
Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference
Model selection in Gaussian processes scales prohibitively with the size of the training dataset, both in time and memory.
A Fast, Robust Elliptical Slice Sampling Implementation for Linearly Truncated Multivariate Normal Distributions
Elliptical slice sampling, when adapted to linearly truncated multivariate normal distributions, is a rejection-free Markov chain Monte Carlo method.
Approximation-Aware Bayesian Optimization
Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design.
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models using only forward passes.
Understanding Stochastic Natural Gradient Variational Inference
For conjugate likelihoods, we prove the first $\mathcal{O}(\frac{1}{T})$ non-asymptotic convergence rate of stochastic NGVI.
Demystifying SGD with Doubly Stochastic Gradients
For these problems, a popular strategy is to employ SGD with doubly stochastic gradients (doubly SGD): the expectations are estimated using the gradient estimator of each component, while the sum is estimated by subsampling over these estimators.
Stochastic Approximation with Biased MCMC for Expectation Maximization
In practice, MCMC-SAEM is often run with asymptotically biased MCMC, for which the consequences are theoretically less understood.
Provably Scalable Black-Box Variational Inference with Structured Variational Families
In fact, recent computational complexity results for BBVI have established that full-rank variational families scale poorly with the dimensionality of the problem compared to e. g. mean-field families.
Large-Scale Gaussian Processes via Alternating Projection
Training and inference in Gaussian processes (GPs) require solving linear systems with $n\times n$ kernel matrices.
Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?
We prove that black-box variational inference (BBVI) with control variates, particularly the sticking-the-landing (STL) estimator, converges at a geometric (traditionally called "linear") rate under perfect variational family specification.
Inverse Protein Folding Using Deep Bayesian Optimization
Furthermore, it is challenging to adapt pure generative approaches to other settings, e. g., when constraints exist.
On the Convergence of Black-Box Variational Inference
We provide the first convergence guarantee for full black-box variational inference (BBVI), also known as Monte Carlo variational inference.
The Behavior and Convergence of Local Bayesian Optimization
A recent development in Bayesian optimization is the use of local optimization strategies, which can deliver strong empirical performance on high-dimensional problems compared to traditional global strategies.
Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference
Understanding the gradient variance of black-box variational inference (BBVI) is a crucial step for establishing its convergence and developing algorithmic improvements.
Local Bayesian optimization via maximizing probability of descent
Local optimization presents a promising approach to expensive, high-dimensional black-box optimization by sidestepping the need to globally explore the search space.
Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction
In this paper, we seek to improve the faithfulness of TempRel extraction models from two perspectives.
Ranked #3 on
Temporal Relation Classification
on MATRES
Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients
Minimizing the inclusive Kullback-Leibler (KL) divergence with stochastic gradient descent (SGD) is challenging since its gradient is defined as an integral over the posterior.
Local Latent Space Bayesian Optimization over Structured Inputs
By reformulating the encoder to function as both an encoder for the DAE globally and as a deep kernel for the surrogate model within a trust region, we better align the notion of local optimization in the latent space with local optimization in the input space.
Scaling Gaussian Processes with Derivative Information Using Variational Inference
We demonstrate the full scalability of our approach on a variety of tasks, ranging from a high dimensional stellarator fusion regression task to training graph convolutional neural networks on Pubmed using Bayesian optimization.
Preconditioning for Scalable Gaussian Process Hyperparameter Optimization
While preconditioning is well understood in the context of CG, we demonstrate that it can also accelerate convergence and reduce variance of the estimates for the log-determinant and its derivative.
Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees
In this paper, we provide the first efficient implementation of general multi-step lookahead Bayesian optimization, formulated as a sequence of nested optimization problems within a multi-step scenario tree.
Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
Matrix square roots and their inverses arise frequently in machine learning, e. g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$.
Deep Sigma Point Processes
We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs).
Parametric Gaussian Process Regressors
In an extensive empirical comparison with a number of alternative methods for scalable GP regression, we find that the resulting predictive distributions exhibit significantly better calibrated uncertainties and higher log likelihoods--often by as much as half a nat per datapoint.
Scalable Global Optimization via Local Bayesian Optimization
This motivates the design of a local probabilistic approach for global optimization of large-scale high-dimensional problems.
Simple Black-box Adversarial Attacks
We propose an intriguingly simple method for the construction of adversarial images in the black-box setting.
Exact Gaussian Processes on a Million Data Points
Gaussian processes (GPs) are flexible non-parametric models, with a capacity that grows with the available data.
GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware.
Constant-Time Predictive Distributions for Gaussian Processes
One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions.
Product Kernel Interpolation for Scalable Gaussian Processes
Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs).
Deep Manifold Traversal: Changing Labels with Convolutional Features
Many tasks in computer vision can be cast as a "label changing" problem, where the goal is to make a semantic change to the appearance of an image or some subject in an image in order to alter the class membership.
Compressed Support Vector Machines
For most of the time during which we conducted this research, we were unaware of this prior work.
Differentially Private Bayesian Optimization
The success of machine learning has led practitioners in diverse real-world settings to learn classifiers for practical problems.
A Reduction of the Elastic Net to Support Vector Machines with an Application to GPU Computing
Our second contribution is to derive a practical algorithm based on this reduction.
Parallel Support Vector Machines in Practice
In particular, we provide the first comparison of algorithms with explicit and implicit parallelization.
