Search Results for author: Willie Neiswanger

Found 53 papers, 28 papers with code

IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations

no code implementations1 Apr 2024 Deqing Fu, Ghazal Khalighinejad, Ollie Liu, Bhuwan Dhingra, Dani Yogatama, Robin Jia, Willie Neiswanger

Current foundation models exhibit impressive capabilities when prompted either with text only or with both image and text inputs.

Benchmarking Math

Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution

2 code implementations13 Feb 2024 Tailin Wu, Willie Neiswanger, Hongtao Zheng, Stefano Ermon, Jure Leskovec

Deep learning-based surrogate models have demonstrated remarkable advantages over classical solvers in terms of speed, often achieving speedups of 10 to 1000 times over traditional partial differential equation (PDE) solvers.

Decision Making Uncertainty Quantification

DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models

no code implementations4 Feb 2024 Ollie Liu, Deqing Fu, Dani Yogatama, Willie Neiswanger

Large language models (LLMs) are increasingly used across society, including in domains like business, engineering, and medicine.

Decision Making Decision Making Under Uncertainty +2

Sample Efficient Reinforcement Learning from Human Feedback via Active Exploration

no code implementations1 Dec 2023 Viraj Mehta, Vikramjeet Das, Ojash Neopane, Yijia Dai, Ilija Bogunovic, Jeff Schneider, Willie Neiswanger

Preference-based feedback is important for many applications in reinforcement learning where direct evaluation of a reward function is not feasible.


SlimPajama-DC: Understanding Data Combinations for LLM Training

no code implementations19 Sep 2023 Zhiqiang Shen, Tianhua Tao, Liqun Ma, Willie Neiswanger, Zhengzhong Liu, Hongyi Wang, Bowen Tan, Joel Hestness, Natalia Vassilieva, Daria Soboleva, Eric Xing

This paper aims to understand the impacts of various data combinations (e. g., web text, wikipedia, github, books) on the training of large language models using SlimPajama.

Kernelized Offline Contextual Dueling Bandits

no code implementations21 Jul 2023 Viraj Mehta, Ojash Neopane, Vikramjeet Das, Sen Lin, Jeff Schneider, Willie Neiswanger

Preference-based feedback is important for many applications where direct evaluation of a reward function is not feasible.

Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching

no code implementations5 Mar 2023 Lantao Yu, Tianhe Yu, Jiaming Song, Willie Neiswanger, Stefano Ermon

In this case, a well-known issue is the distribution shift between the learned policy and the behavior policy that collects the offline data.

Continuous Control Imitation Learning

Near-optimal Policy Identification in Active Reinforcement Learning

no code implementations19 Dec 2022 Xiang Li, Viraj Mehta, Johannes Kirschner, Ian Char, Willie Neiswanger, Jeff Schneider, Andreas Krause, Ilija Bogunovic

Many real-world reinforcement learning tasks require control of complex dynamical systems that involve both costly data acquisition processes and large state spaces.

Bayesian Optimization reinforcement-learning +1

Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis

1 code implementation10 Oct 2022 Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, Louis-Philippe Morency

In particular, there are various considerations behind the pipeline: (1) the choice and (2) the size of PLM, (3) the choice of uncertainty quantifier, (4) the choice of fine-tuning loss, and many more.

Uncertainty Quantification

AutoML for Climate Change: A Call to Action

1 code implementation7 Oct 2022 Renbo Tu, Nicholas Roberts, Vishak Prasad, Sibasis Nayak, Paarth Jain, Frederic Sala, Ganesh Ramakrishnan, Ameet Talwalkar, Willie Neiswanger, Colin White

The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications.


Exploration via Planning for Information about the Optimal Trajectory

1 code implementation6 Oct 2022 Viraj Mehta, Ian Char, Joseph Abbate, Rory Conlin, Mark D. Boyer, Stefano Ermon, Jeff Schneider, Willie Neiswanger

In this work, we develop a method that allows us to plan for exploration while taking both the task and the current knowledge about the dynamics into account.

Reinforcement Learning (RL)

Generalizing Bayesian Optimization with Decision-theoretic Entropies

no code implementations4 Oct 2022 Willie Neiswanger, Lantao Yu, Shengjia Zhao, Chenlin Meng, Stefano Ermon

Bayesian optimization (BO) is a popular method for efficiently inferring optima of an expensive black-box function via a sequence of queries.

Bayesian Optimization Decision Making

Multipoint-BAX: A New Approach for Efficiently Tuning Particle Accelerator Emittance via Virtual Objectives

no code implementations10 Sep 2022 Sara A. Miskovich, Willie Neiswanger, William Colocho, Claudio Emma, Jacqueline Garrahan, Timothy Maxwell, Christopher Mayes, Stefano Ermon, Auralee Edelen, Daniel Ratner

Traditional black-box optimizers such as Bayesian optimization are slow and inefficient when dealing with such objectives as they must acquire the full series of measurements, but return only the emittance, with each query.

Bayesian Optimization

Betty: An Automatic Differentiation Library for Multilevel Optimization

1 code implementation5 Jul 2022 Sang Keun Choe, Willie Neiswanger, Pengtao Xie, Eric Xing

Gradient-based multilevel optimization (MLO) has gained attention as a framework for studying numerous problems, ranging from hyperparameter optimization and meta-learning to neural architecture search and reinforcement learning.

Hyperparameter Optimization Meta-Learning +1

A General Recipe for Likelihood-free Bayesian Optimization

1 code implementation27 Jun 2022 Jiaming Song, Lantao Yu, Willie Neiswanger, Stefano Ermon

To extend BO to a broader class of models and utilities, we propose likelihood-free BO (LFBO), an approach based on likelihood-free inference.

Bayesian Optimization

Modular Conformal Calibration

no code implementations23 Jun 2022 Charles Marx, Shengjia Zhao, Willie Neiswanger, Stefano Ermon

We introduce a versatile class of algorithms for recalibration in regression that we call Modular Conformal Calibration (MCC).


Generative Modeling Helps Weak Supervision (and Vice Versa)

1 code implementation22 Mar 2022 Benedikt Boecking, Nicholas Roberts, Willie Neiswanger, Stefano Ermon, Frederic Sala, Artur Dubrawski

The model outperforms baseline weak supervision label models on a number of multiclass image classification datasets, improves the quality of generated images, and further improves end-model performance through data augmentation with synthetic samples.

Data Augmentation Image Classification

IS-COUNT: Large-scale Object Counting from Satellite Images with Covariate-based Importance Sampling

1 code implementation16 Dec 2021 Chenlin Meng, Enci Liu, Willie Neiswanger, Jiaming Song, Marshall Burke, David Lobell, Stefano Ermon

We show empirically that the proposed framework achieves strong performance on estimating the number of buildings in the United States and Africa, cars in Kenya, brick kilns in Bangladesh, and swimming pools in the U. S., while requiring as few as 0. 01% of satellite images compared to an exhaustive approach.

Object Object Counting +2

An Experimental Design Perspective on Model-Based Reinforcement Learning

1 code implementation9 Dec 2021 Viraj Mehta, Biswajit Paria, Jeff Schneider, Stefano Ermon, Willie Neiswanger

In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.

Continuous Control Experimental Design +3

Personalized Benchmarking with the Ludwig Benchmarking Toolkit

2 code implementations8 Nov 2021 Avanika Narayan, Piero Molino, Karan Goel, Willie Neiswanger, Christopher Ré

LBT provides a configurable interface for controlling training and customizing evaluation, a standardized training framework for eliminating confounding variables, and support for multi-objective evaluation.

Benchmarking Hyperparameter Optimization +2

An Experimental Design Perspective on Exploration in Reinforcement Learning

no code implementations ICLR 2022 Viraj Mehta, Biswajit Paria, Jeff Schneider, Willie Neiswanger, Stefano Ermon

In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.

Continuous Control Experimental Design +2

H-Entropy Search: Generalizing Bayesian Optimization with a Decision-theoretic Uncertainty Measure

no code implementations29 Sep 2021 Willie Neiswanger, Lantao Yu, Shengjia Zhao, Chenlin Meng, Stefano Ermon

For special cases of the loss and design space, we develop gradient-based methods to efficiently optimize our proposed family of acquisition functions, and demonstrate that the resulting BO procedure shows strong empirical performance on a diverse set of optimization tasks.

Bayesian Optimization

Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification

1 code implementation21 Sep 2021 Youngseog Chung, Ian Char, Han Guo, Jeff Schneider, Willie Neiswanger

With increasing deployment of machine learning systems in various real-world tasks, there is a greater need for accurate quantification of predictive uncertainty.

BIG-bench Machine Learning Uncertainty Quantification

Synthetic Benchmarks for Scientific Research in Explainable Machine Learning

1 code implementation23 Jun 2021 Yang Liu, Sujay Khandagale, Colin White, Willie Neiswanger

In this work, we address this issue by releasing XAI-Bench: a suite of synthetic datasets along with a library for benchmarking feature attribution algorithms.

Benchmarking BIG-bench Machine Learning +1

Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation

1 code implementation17 Jun 2021 Yuxin Xiao, Eric P. Xing, Willie Neiswanger

With the surge in the number of hyperparameters and training times of modern machine learning models, hyperparameter tuning is becoming increasingly expensive.

Bayesian Optimization Transfer Learning

Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information

1 code implementation19 Apr 2021 Willie Neiswanger, Ke Alexander Wang, Stefano Ermon

Given such an $\mathcal{A}$, and a prior distribution over $f$, we refer to the problem of inferring the output of $\mathcal{A}$ using $T$ evaluations as Bayesian Algorithm Execution (BAX).

Bayesian Optimization Experimental Design +1

Computational catalyst discovery: Active classification through myopic multiscale sampling

no code implementations2 Feb 2021 Kevin Tran, Willie Neiswanger, Kirby Broderick, Erix Xing, Jeff Schneider, Zachary W. Ulissi

We address this issue by relaxing the catalyst discovery goal into a classification problem: "What is the set of catalysts that is worth testing experimentally?"

Chemical Physics

Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling

1 code implementation ICLR 2021 Benedikt Boecking, Willie Neiswanger, Eric Xing, Artur Dubrawski

Our experiments demonstrate that only a small number of feedback iterations are needed to train models that achieve highly competitive test set performance without access to ground truth training labels.

Weakly Supervised Classification

Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification

2 code implementations NeurIPS 2021 Youngseog Chung, Willie Neiswanger, Ian Char, Jeff Schneider

However, this loss restricts the scope of applicable regression models, limits the ability to target many desirable properties (e. g. calibration, sharpness, centered intervals), and may produce poor conditional quantiles.

regression Uncertainty Quantification

Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning

2 code implementations27 Aug 2020 Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, Eric P. Xing

Some recent schedulers choose job resources for users, but do so without awareness of how DL training can be re-optimized to better utilize the provided resources.

Fairness Scheduling

A Study on Encodings for Neural Architecture Search

2 code implementations NeurIPS 2020 Colin White, Willie Neiswanger, Sam Nolen, Yash Savani

First we formally define architecture encodings and give a theoretical characterization on the scalability of the encodings we study Then we identify the main encoding-dependent subroutines which NAS algorithms employ, running experiments to show which encodings work best with each subroutine for many popular algorithms.

Neural Architecture Search

Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Prediction

no code implementations23 Jun 2020 Viraj Mehta, Ian Char, Willie Neiswanger, Youngseog Chung, Andrew Oakleigh Nelson, Mark D Boyer, Egemen Kolemen, Jeff Schneider

We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential equations.

Uncertainty quantification using martingales for misspecified Gaussian processes

1 code implementation12 Jun 2020 Willie Neiswanger, Aaditya Ramdas

There is a necessary cost to achieving robustness: if the prior was correct, posterior GP bands are narrower than our CS.

Bayesian Optimization Gaussian Processes +2

Neural Dynamical Systems

no code implementations ICLR Workshop DeepDiffEq 2019 Viraj Mehta, Ian Char, Willie Neiswanger, Youngseog Chung, Andrew Oakleigh Nelson, Mark D Boyer, Egemen Kolemen, Jeff Schneider

We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models which incorporates prior knowledge in the form of systems of ordinary differential equations.

Methods for comparing uncertainty quantifications for material property predictions

1 code implementation20 Dec 2019 Kevin Tran, Willie Neiswanger, Junwoong Yoon, Eric Xing, Zachary W. Ulissi

These uncertainty estimates are instrumental for determining which materials to screen next, but there is not yet a standard procedure for judging the quality of such uncertainty estimates objectively.

Materials Science Computational Physics

Offline Contextual Bayesian Optimization

1 code implementation NeurIPS 2019 Ian Char, Youngseog Chung, Willie Neiswanger, Kirthevasan Kandasamy, Oak Nelson, Mark Boyer, Egemen Kolemen

In black-box optimization, an agent repeatedly chooses a configuration to test, so as to find an optimal configuration.

Bayesian Optimization

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search

3 code implementations25 Oct 2019 Colin White, Willie Neiswanger, Yash Savani

Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very promising strategy for NAS when it is coupled with a neural predictor.

Bayesian Optimization Hyperparameter Optimization +1

BANANAS: Bayesian Optimization with Neural Networks for Neural Architecture Search

no code implementations25 Sep 2019 Colin White, Willie Neiswanger, Yash Savani

We develop a path-based encoding scheme to featurize the neural architectures that are used to train the neural network model.

Bayesian Optimization Neural Architecture Search +2

ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations

1 code implementation5 Aug 2019 Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, Eric P. Xing

In applications such as molecule design or drug discovery, it is desirable to have an algorithm which recommends new candidate molecules based on the results of past tests.

Bayesian Optimization Drug Discovery

Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly

1 code implementation15 Mar 2019 Kirthevasan Kandasamy, Karun Raju Vysyaraju, Willie Neiswanger, Biswajit Paria, Christopher R. Collins, Jeff Schneider, Barnabas Poczos, Eric P. Xing

We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO.

Bayesian Optimisation

ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language

1 code implementation31 Jan 2019 Willie Neiswanger, Kirthevasan Kandasamy, Barnabas Poczos, Jeff Schneider, Eric Xing

Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum.

Bayesian Optimization Gaussian Processes +1

Geometric Generalization Based Zero-Shot Learning Dataset Infinite World: Simple Yet Powerful

no code implementations10 Jul 2018 Rajesh Chidambaram, Michael Kampffmeyer, Willie Neiswanger, Xiaodan Liang, Thomas Lachmann, Eric Xing

Analogously, this paper introduces geometric generalization based zero-shot learning tests to measure the rapid learning ability and the internal consistency of deep generative models.

Zero-Shot Learning

Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming

1 code implementation25 May 2018 Kirthevasan Kandasamy, Willie Neiswanger, Reed Zhang, Akshay Krishnamurthy, Jeff Schneider, Barnabas Poczos

We design a new myopic strategy for a wide class of sequential design of experiment (DOE) problems, where the goal is to collect data in order to to fulfil a certain problem specific goal.

Multi-Armed Bandits Probabilistic Programming +2

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

1 code implementation NeurIPS 2018 Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, Eric Xing

A common use case for BO in machine learning is model selection, where it is not possible to analytically model the generalisation performance of a statistical model, and we resort to noisy and expensive training and validation procedures to choose the best model.

Bayesian Optimisation BIG-bench Machine Learning +2

Performance Bounds for Graphical Record Linkage

no code implementations8 Mar 2017 Rebecca C. Steorts, Matt Barnes, Willie Neiswanger

Record linkage involves merging records in large, noisy databases to remove duplicate entities.


Post-Inference Prior Swapping

no code implementations ICML 2017 Willie Neiswanger, Eric Xing

However, we demonstrate that IS will fail for many choices of the target prior, depending on its parametric form and similarity to the false prior.

Embarrassingly Parallel Variational Inference in Nonconjugate Models

no code implementations14 Oct 2015 Willie Neiswanger, Chong Wang, Eric Xing

We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines.

Variational Inference

Fast Function to Function Regression

no code implementations27 Oct 2014 Junier Oliva, Willie Neiswanger, Barnabas Poczos, Eric Xing, Jeff Schneider

Function to function regression (FFR) covers a large range of interesting applications including time-series prediction problems, and also more general tasks like studying a mapping between two separate types of distributions.

regression Time Series +1

Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms

no code implementations22 Sep 2014 Yu-Xiang Wang, Veeranjaneyulu Sadhanala, Wei Dai, Willie Neiswanger, Suvrit Sra, Eric P. Xing

We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework.

Asymptotically Exact, Embarrassingly Parallel MCMC

no code implementations19 Nov 2013 Willie Neiswanger, Chong Wang, Eric Xing

This embarrassingly parallel algorithm allows each machine to act independently on a subset of the data (without communication) until the final combination stage.

Fast Distribution To Real Regression

no code implementations10 Nov 2013 Junier B. Oliva, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, Eric Xing

We study the problem of distribution to real-value regression, where one aims to regress a mapping $f$ that takes in a distribution input covariate $P\in \mathcal{I}$ (for a non-parametric family of distributions $\mathcal{I}$) and outputs a real-valued response $Y=f(P) + \epsilon$.


Cannot find the paper you are looking for? You can Submit a new open access paper.