Search Results for author:
Found 64 papers, 35 papers with code
Debiased Distribution Compression
Modern compression methods can summarize a target distribution $\mathbb{P}$ more succinctly than i. i. d.
SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery
The resulting SatCLIP location encoder efficiently summarizes the characteristics of any given location for convenient use in downstream tasks.
Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
In this work, we use a language-model-infused scaffolding program to improve itself.
Reflections from the Workshop on AI-Assisted Decision Making for Conservation
In this white paper, we synthesize key points made during presentations and discussions from the AI-Assisted Decision Making for Conservation workshop, hosted by the Center for Research on Computation and Society at Harvard University on October 20-21, 2022.
Do Language Models Know When They're Hallucinating References?
In this work, we focus on hallucinated book and article references and present them as the "model organism" of language model hallucination research, due to their frequent and easy-to-discern nature.
Learning Rate Free Sampling in Constrained Domains
We introduce a suite of new particle-based algorithms for sampling in constrained domains which are entirely learning rate free.
Compress Then Test: Powerful Kernel Testing in Near-linear Time
To address these shortcomings, we introduce Compress Then Test (CTT), a new framework for high-powered kernel testing based on sample compression.
Controlling Moments with Kernel Stein Discrepancies
Kernel Stein discrepancies (KSDs) measure the quality of a distributional approximation and can be computed even when the target density has an intractable normalizing constant.
Budget-Constrained Bounds for Mini-Batch Estimation of Optimal Transport
Optimal Transport (OT) is a fundamental tool for comparing probability distributions, but its exact computation remains prohibitive for large datasets.
Targeted Separation and Convergence with Kernel Discrepancies
Maximum mean discrepancies (MMDs) like the kernel Stein discrepancy (KSD) have grown central to a wide range of applications, including hypothesis testing, sampler selection, distribution approximation, and variational inference.
Adaptive Bias Correction for Improved Subseasonal Forecasting
Subseasonal forecasting -- predicting temperature and precipitation 2 to 6 weeks ahead -- is critical for effective water allocation, wildfire management, and drought and flood mitigation.
Scalable Spike-and-Slab
Spike-and-slab priors are commonly used for Bayesian variable selection, due to their interpretability and favorable statistical properties.
Gradient Estimation with Discrete Stein Operators
Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems.
Bounding Wasserstein distance with couplings
Markov chain Monte Carlo (MCMC) provides asymptotically consistent estimates of intractable posterior expectations as the number of iterations tends to infinity.
Distribution Compression in Near-linear Time
Near-optimal thinning procedures achieve this goal by sampling $n$ points from a Markov chain and identifying $\sqrt{n}$ points with $\widetilde{\mathcal{O}}(1/\sqrt{n})$ discrepancy to $\mathbb{P}$.
Generalized Kernel Thinning
Fourth, we establish that KT applied to a sum of the target and power kernels (a procedure we call KT+) simultaneously inherits the improved MMD guarantees of power KT and the tighter individual function guarantees of target KT.
SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking
To streamline this process and accelerate future development, we introduce SubseasonalClimateUSA, a curated dataset for training and benchmarking subseasonal forecasting models in the United States.
Social Norm Bias: Residual Harms of Fairness-Aware Algorithms
Many modern machine learning algorithms mitigate bias by enforcing fairness constraints across coarsely-defined groups related to a sensitive attribute like gender or race.
Near-optimal inference in adaptive linear regression
We propose a family of online debiasing estimators to correct these distributional anomalies in least squares estimation.
Sampling with Mirrored Stein Operators
We introduce a new family of particle evolution samplers suitable for constrained domains and non-Euclidean geometries.
Online Learning with Optimism and Delay
Inspired by the demands of real-time climate and weather forecasting, we develop optimistic online learning algorithms that require no parameter tuning and have optimal regret guarantees under delayed feedback.
Kernel Thinning
The maximum discrepancy in integration error is $\mathcal{O}_d(n^{-1/2}\sqrt{\log n})$ in probability for compactly supported $\mathbb{P}$ and $\mathcal{O}_d(n^{-\frac{1}{2}} (\log n)^{(d+1)/2}\sqrt{\log\log n})$ for sub-exponential $\mathbb{P}$ on $\mathbb{R}^d$.
Initialization and Regularization of Factorized Neural Layers
In model compression, we show that they enable low-rank methods to significantly outperform both unstructured sparsity and tensor methods on the task of training low-memory residual networks; analogs of the schemes also improve the performance of tensor decomposition techniques.
Knowledge Distillation as Semiparametric Inference
A popular approach to model compression is to train an inexpensive student model to mimic the class probabilities of a highly accurate but cumbersome teacher model.
Model-specific Data Subsampling with Influence Functions
Model selection requires repeatedly evaluating models on a given dataset and measuring their relative performances.
Independent versus truncated finite approximations for Bayesian nonparametric inference
Bayesian nonparametric models based on completely random measures (CRMs) offers flexibility when the number of clusters or latent components in a data set is unknown.
Independent finite approximations for Bayesian nonparametric inference
We call our construction the automated independent finite approximation (AIFA).
Cross-validation Confidence Intervals for Test Error
This work develops central limit theorems for cross-validation and consistent estimators of its asymptotic variance under weak stability conditions on the learning algorithm.
Stochastic Stein Discrepancies
Stein discrepancies (SDs) monitor convergence and non-convergence in approximate inference when exact integration and sampling are intractable.
Metrizing Weak Convergence with Maximum Mean Discrepancies
More precisely, we prove that, on a locally compact, non-compact, Hausdorff space, the MMD of a bounded continuous Borel measurable kernel k, whose reproducing kernel Hilbert space (RKHS) functions vanish at infinity, metrizes the weak convergence of probability measures if and only if k is continuous and integrally strictly positive definite (i. s. p. d.)
Minimax Estimation of Conditional Moment Models
We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression.
Optimal Thinning of MCMC Output
The use of heuristics to assess the convergence and compress the output of Markov chain Monte Carlo can be sub-optimal in terms of the empirical approximations that are produced.
Weighted Meta-Learning
Meta-learning leverages related source tasks to learn an initialization that can be quickly fine-tuned to a target task with limited labeled examples.
Approximate Cross-validation: Guarantees for Model Assessment and Selection
Cross-validation (CV) is a popular approach for assessing and selecting predictive models.
Importance Sampling via Local Sensitivity
Unfortunately, sensitivity sampling is difficult to apply since (1) it is unclear how to efficiently compute the sensitivity scores and (2) the sample size required is often impractically large.
Single Point Transductive Prediction
Standard methods in supervised learning separate training and prediction: the model is fit independently of any test points it may encounter.
A Kernel Stein Test for Comparing Latent Variable Models
We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable.
Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond
In this paper, we establish the convergence rate of sampling algorithms obtained by discretizing smooth It\^o diffusions exhibiting fast Wasserstein-$2$ contraction, based on local deviation properties of the integration scheme.
Minimum Stein Discrepancy Estimators
We provide a unifying perspective of these techniques as minimum Stein discrepancy estimators, and use this lens to design new diffusion kernel Stein discrepancy (DKSD) and diffusion score matching (DSM) estimators with complementary strengths.
Stein Point Markov Chain Monte Carlo
Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target and, hence, require the solution of a non-convex optimisation problem to obtain each new point.
Model Compression with Generative Adversarial Networks
Our GAN-assisted model compression (GAN-MC) significantly improves student accuracy for expensive models such as deep neural networks and large random forests on both image and tabular datasets.
Accelerating Rescaled Gradient Descent: Fast Optimization of Smooth Functions
We also introduce a new first-order algorithm, called rescaled gradient descent (RGD), and show that RGD achieves a faster convergence rate than gradient descent provided the function is strongly smooth -- a natural generalization of the standard smoothness assumption on the objective function.
Optimization and Control
Teacher-Student Compression with Generative Adversarial Networks
Our GAN-assisted TSC (GAN-TSC) significantly improves student accuracy for expensive models such as large random forests and deep neural networks on both tabular and image datasets.
Global Non-convex Optimization with Discretized Diffusions
An Euler discretization of the Langevin diffusion is known to converge to the global minimizers of certain convex and non-convex optimization problems.
Improving Subseasonal Forecasting in the Western U.S. with Machine Learning
We hope that both our dataset and our methods will help to advance the state of the art in subseasonal forecasting.
Random Feature Stein Discrepancies
Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing.
DeepMiner: Discovering Interpretable Representations for Mammogram Classification and Explanation
We propose DeepMiner, a framework to discover interpretable representations in deep neural networks and to build explanations for medical predictions.
Stein Points
An important task in computational statistics and machine learning is to approximate a posterior distribution $p(x)$ with an empirical measure supported on a set of representative points $\{x_i\}_{i=1}^n$.
Expert identification of visual primitives used by CNNs during mammogram classification
This work interprets the internal representations of deep neural networks trained for classification of diseased tissue in 2D mammograms.
Accurate Inference for Adaptive Linear Models
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren.
Orthogonal Machine Learning: Power and Limitations
Double machine learning provides $\sqrt{n}$-consistent estimates of parameters of interest even when high-dimensional or nonparametric nuisance parameters are estimated at an $n^{-1/4}$ rate.
Improving Gibbs Sampler Scan Quality with DoGS
The pairwise influence matrix of Dobrushin has long been used as an analytical tool to bound the rate of convergence of Gibbs sampling.
Measuring Sample Quality with Kernels
We develop a theory of weak convergence for KSDs based on Stein's method, demonstrate that commonly used KSDs fail to detect non-convergence even for Gaussian targets, and show that kernels with slowly decaying tails provably determine convergence for a large class of target distributions.
Measuring Sample Quality with Diffusions
Stein's method for measuring convergence to a continuous target distribution relies on an operator characterizing the target and Stein factor bounds on the solutions of an associated differential equation.
Jet-Images -- Deep Learning Edition
Building on the notion of a particle physics detector as a camera and the collimated streams of high energy particles, or jets, it measures as an image, we investigate the potential of machine learning techniques based on deep learning architectures to identify highly boosted W bosons.
Fuzzy Jets
Collimated streams of particles produced in high energy physics experiments are organized using clustering algorithms to form jets.
Measuring Sample Quality with Stein's Method
To improve the efficiency of Monte Carlo estimation, practitioners are turning to biased Markov chain Monte Carlo procedures that trade off asymptotic exactness for computational speed.
Weighted Classification Cascades for Optimizing Discovery Significance in the HiggsML Challenge
We introduce a minorization-maximization approach to optimizing common measures of discovery significance in high energy physics.
Corrupted Sensing: Novel Guarantees for Separating Structured Signals
While an arbitrary signal cannot be recovered in the face of arbitrary corruption, tractable recovery is possible when both signal and corruption are suitably structured.
Distributed Low-rank Subspace Segmentation
Vision problems ranging from image clustering to motion segmentation to semi-supervised learning can naturally be framed as subspace segmentation problems, in which one aims to recover multiple low-dimensional subspaces from noisy and corrupted input data.
The asymptotics of ranking algorithms
With these negative results as motivation, we present a new approach to supervised ranking based on aggregation of partial preferences, and we develop $U$-statistic-based empirical risk minimization procedures.
Combinatorial clustering and the beta negative binomial process
We show that the NBP is conjugate to the beta process, and we characterize the posterior distribution under the beta-negative binomial process (BNBP) and hierarchical models based on the BNBP (the HBNBP).
Distributed Matrix Completion and Robust Factorization
If learning methods are to scale to the massive sizes of modern datasets, it is essential for the field of machine learning to embrace parallel and distributed computing.
Feature-Weighted Linear Stacking
Ensemble methods, such as stacking, are designed to boost predictive accuracy by blending the predictions of multiple machine learning models.
