Search Results for author: Willie Neiswanger

Found 38 papers, 20 papers with code

A General Recipe for Likelihood-free Bayesian Optimization

1 code implementation27 Jun 2022 Jiaming Song, Lantao Yu, Willie Neiswanger, Stefano Ermon

To extend BO to a broader class of models and utilities, we propose likelihood-free BO (LFBO), an approach based on likelihood-free inference.

Modular Conformal Calibration

no code implementations23 Jun 2022 Charles Marx, Shengjia Zhou, Willie Neiswanger, Stefano Ermon

We introduce a versatile class of algorithms for recalibration in regression that we call Modular Conformal Calibration (MCC).

Generative Modeling Helps Weak Supervision (and Vice Versa)

no code implementations22 Mar 2022 Benedikt Boecking, Nicholas Roberts, Willie Neiswanger, Stefano Ermon, Frederic Sala, Artur Dubrawski

The model outperforms baseline weak supervision label models on a number of multiclass image classification datasets, improves the quality of generated images, and further improves end-model performance through data augmentation with synthetic samples.

Data Augmentation Image Classification

IS-COUNT: Large-scale Object Counting from Satellite Images with Covariate-based Importance Sampling

1 code implementation16 Dec 2021 Chenlin Meng, Enci Liu, Willie Neiswanger, Jiaming Song, Marshall Burke, David Lobell, Stefano Ermon

We show empirically that the proposed framework achieves strong performance on estimating the number of buildings in the United States and Africa, cars in Kenya, brick kilns in Bangladesh, and swimming pools in the U. S., while requiring as few as 0. 01% of satellite images compared to an exhaustive approach.

Object Counting object-detection +1

An Experimental Design Perspective on Model-Based Reinforcement Learning

no code implementations9 Dec 2021 Viraj Mehta, Biswajit Paria, Jeff Schneider, Stefano Ermon, Willie Neiswanger

In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.

Continuous Control Experimental Design +2

Personalized Benchmarking with the Ludwig Benchmarking Toolkit

2 code implementations8 Nov 2021 Avanika Narayan, Piero Molino, Karan Goel, Willie Neiswanger, Christopher Ré

LBT provides a configurable interface for controlling training and customizing evaluation, a standardized training framework for eliminating confounding variables, and support for multi-objective evaluation.

Hyperparameter Optimization Text Classification

An Experimental Design Perspective on Exploration in Reinforcement Learning

no code implementations ICLR 2022 Viraj Mehta, Biswajit Paria, Jeff Schneider, Willie Neiswanger, Stefano Ermon

In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.

Continuous Control Experimental Design +1

H-Entropy Search: Generalizing Bayesian Optimization with a Decision-theoretic Uncertainty Measure

no code implementations29 Sep 2021 Willie Neiswanger, Lantao Yu, Shengjia Zhao, Chenlin Meng, Stefano Ermon

For special cases of the loss and design space, we develop gradient-based methods to efficiently optimize our proposed family of acquisition functions, and demonstrate that the resulting BO procedure shows strong empirical performance on a diverse set of optimization tasks.

Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification

1 code implementation21 Sep 2021 Youngseog Chung, Ian Char, Han Guo, Jeff Schneider, Willie Neiswanger

With increasing deployment of machine learning systems in various real-world tasks, there is a greater need for accurate quantification of predictive uncertainty.

Synthetic Benchmarks for Scientific Research in Explainable Machine Learning

1 code implementation23 Jun 2021 Yang Liu, Sujay Khandagale, Colin White, Willie Neiswanger

In this work, we address this issue by releasing XAI-Bench: a suite of synthetic datasets along with a library for benchmarking feature attribution algorithms.

Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation

1 code implementation17 Jun 2021 Yuxin Xiao, Eric P. Xing, Willie Neiswanger

With the surge in the number of hyperparameters and training times of modern machine learning models, hyperparameter tuning is becoming increasingly expensive.

Transfer Learning

Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information

1 code implementation19 Apr 2021 Willie Neiswanger, Ke Alexander Wang, Stefano Ermon

Given such an $\mathcal{A}$, and a prior distribution over $f$, we refer to the problem of inferring the output of $\mathcal{A}$ using $T$ evaluations as Bayesian Algorithm Execution (BAX).

Experimental Design Gaussian Processes

Computational catalyst discovery: Active classification through myopic multiscale sampling

no code implementations2 Feb 2021 Kevin Tran, Willie Neiswanger, Kirby Broderick, Erix Xing, Jeff Schneider, Zachary W. Ulissi

We address this issue by relaxing the catalyst discovery goal into a classification problem: "What is the set of catalysts that is worth testing experimentally?"

Chemical Physics

Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling

1 code implementation ICLR 2021 Benedikt Boecking, Willie Neiswanger, Eric Xing, Artur Dubrawski

Our experiments demonstrate that only a small number of feedback iterations are needed to train models that achieve highly competitive test set performance without access to ground truth training labels.

Weakly Supervised Classification

Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification

2 code implementations NeurIPS 2021 Youngseog Chung, Willie Neiswanger, Ian Char, Jeff Schneider

However, this loss restricts the scope of applicable regression models, limits the ability to target many desirable properties (e. g. calibration, sharpness, centered intervals), and may produce poor conditional quantiles.

Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning

2 code implementations27 Aug 2020 Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, Eric P. Xing

Some recent schedulers choose job resources for users, but do so without awareness of how DL training can be re-optimized to better utilize the provided resources.

Fairness

A Study on Encodings for Neural Architecture Search

2 code implementations NeurIPS 2020 Colin White, Willie Neiswanger, Sam Nolen, Yash Savani

First we formally define architecture encodings and give a theoretical characterization on the scalability of the encodings we study Then we identify the main encoding-dependent subroutines which NAS algorithms employ, running experiments to show which encodings work best with each subroutine for many popular algorithms.

Neural Architecture Search

Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction

no code implementations23 Jun 2020 Viraj Mehta, Ian Char, Willie Neiswanger, Youngseog Chung, Andrew Oakleigh Nelson, Mark D Boyer, Egemen Kolemen, Jeff Schneider

We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential equations.

Uncertainty quantification using martingales for misspecified Gaussian processes

1 code implementation12 Jun 2020 Willie Neiswanger, Aaditya Ramdas

There is a necessary cost to achieving robustness: if the prior was correct, posterior GP bands are narrower than our CS.

Gaussian Processes

Neural Dynamical Systems

no code implementations ICLR Workshop DeepDiffEq 2019 Viraj Mehta, Ian Char, Willie Neiswanger, Youngseog Chung, Andrew Oakleigh Nelson, Mark D Boyer, Egemen Kolemen, Jeff Schneider

We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models which incorporates prior knowledge in the form of systems of ordinary differential equations.

Methods for comparing uncertainty quantifications for material property predictions

1 code implementation20 Dec 2019 Kevin Tran, Willie Neiswanger, Junwoong Yoon, Eric Xing, Zachary W. Ulissi

These uncertainty estimates are instrumental for determining which materials to screen next, but there is not yet a standard procedure for judging the quality of such uncertainty estimates objectively.

Materials Science Computational Physics

Offline Contextual Bayesian Optimization

1 code implementation NeurIPS 2019 Ian Char, Youngseog Chung, Willie Neiswanger, Kirthevasan Kandasamy, Oak Nelson, Mark Boyer, Egemen Kolemen

In black-box optimization, an agent repeatedly chooses a configuration to test, so as to find an optimal configuration.

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search

3 code implementations25 Oct 2019 Colin White, Willie Neiswanger, Yash Savani

Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very promising strategy for NAS when it is coupled with a neural predictor.

Hyperparameter Optimization Neural Architecture Search

BANANAS: Bayesian Optimization with Neural Networks for Neural Architecture Search

no code implementations25 Sep 2019 Colin White, Willie Neiswanger, Yash Savani

We develop a path-based encoding scheme to featurize the neural architectures that are used to train the neural network model.

Neural Architecture Search reinforcement-learning

ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations

2 code implementations5 Aug 2019 Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, Eric P. Xing

In applications such as molecule design or drug discovery, it is desirable to have an algorithm which recommends new candidate molecules based on the results of past tests.

Drug Discovery

Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly

1 code implementation15 Mar 2019 Kirthevasan Kandasamy, Karun Raju Vysyaraju, Willie Neiswanger, Biswajit Paria, Christopher R. Collins, Jeff Schneider, Barnabas Poczos, Eric P. Xing

We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO.

Bayesian Optimisation

ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language

1 code implementation31 Jan 2019 Willie Neiswanger, Kirthevasan Kandasamy, Barnabas Poczos, Jeff Schneider, Eric Xing

Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum.

Gaussian Processes Probabilistic Programming

Geometric Generalization Based Zero-Shot Learning Dataset Infinite World: Simple Yet Powerful

no code implementations10 Jul 2018 Rajesh Chidambaram, Michael Kampffmeyer, Willie Neiswanger, Xiaodan Liang, Thomas Lachmann, Eric Xing

Analogously, this paper introduces geometric generalization based zero-shot learning tests to measure the rapid learning ability and the internal consistency of deep generative models.

Zero-Shot Learning

Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming

1 code implementation25 May 2018 Kirthevasan Kandasamy, Willie Neiswanger, Reed Zhang, Akshay Krishnamurthy, Jeff Schneider, Barnabas Poczos

We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal.

Multi-Armed Bandits Probabilistic Programming +1

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

1 code implementation NeurIPS 2018 Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, Eric Xing

A common use case for BO in machine learning is model selection, where it is not possible to analytically model the generalisation performance of a statistical model, and we resort to noisy and expensive training and validation procedures to choose the best model.

Bayesian Optimisation Model Selection +1

Performance Bounds for Graphical Record Linkage

no code implementations8 Mar 2017 Rebecca C. Steorts, Matt Barnes, Willie Neiswanger

Record linkage involves merging records in large, noisy databases to remove duplicate entities.

Post-Inference Prior Swapping

no code implementations ICML 2017 Willie Neiswanger, Eric Xing

However, we demonstrate that IS will fail for many choices of the target prior, depending on its parametric form and similarity to the false prior.

Embarrassingly Parallel Variational Inference in Nonconjugate Models

no code implementations14 Oct 2015 Willie Neiswanger, Chong Wang, Eric Xing

We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines.

Variational Inference

Fast Function to Function Regression

no code implementations27 Oct 2014 Junier Oliva, Willie Neiswanger, Barnabas Poczos, Eric Xing, Jeff Schneider

Function to function regression (FFR) covers a large range of interesting applications including time-series prediction problems, and also more general tasks like studying a mapping between two separate types of distributions.

Time Series Time Series Prediction

Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms

no code implementations22 Sep 2014 Yu-Xiang Wang, Veeranjaneyulu Sadhanala, Wei Dai, Willie Neiswanger, Suvrit Sra, Eric P. Xing

We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework.

Asymptotically Exact, Embarrassingly Parallel MCMC

no code implementations19 Nov 2013 Willie Neiswanger, Chong Wang, Eric Xing

This embarrassingly parallel algorithm allows each machine to act independently on a subset of the data (without communication) until the final combination stage.

Fast Distribution To Real Regression

no code implementations10 Nov 2013 Junier B. Oliva, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, Eric Xing

We study the problem of distribution to real-value regression, where one aims to regress a mapping $f$ that takes in a distribution input covariate $P\in \mathcal{I}$ (for a non-parametric family of distributions $\mathcal{I}$) and outputs a real-valued response $Y=f(P) + \epsilon$.

Cannot find the paper you are looking for? You can Submit a new open access paper.