Search Results for author: Rebecca Willett

Found 51 papers, 15 papers with code

Building a stable classifier with the inflated argmax

1 code implementation22 May 2024 Jake A. Soloff, Rina Foygel Barber, Rebecca Willett

We propose a new framework for algorithmic stability in the context of multiclass classification.

Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet

no code implementations21 May 2024 Melissa Adrian, Daniel Sanz-Alonso, Rebecca Willett

Modern data-driven surrogate models for weather forecasting provide accurate short-term predictions but inaccurate and nonphysical long-term forecasts.

Weather Forecasting

Residual Connections Harm Self-Supervised Abstract Feature Learning

no code implementations16 Apr 2024 Xiao Zhang, Ruoxi Jiang, William Gao, Rebecca Willett, Michael Maire

We demonstrate that adding a weighting factor to decay the strength of identity shortcuts within residual networks substantially improves semantic feature learning in the state-of-the-art self-supervised masked autoencoding (MAE) paradigm.

Depth Separation in Norm-Bounded Infinite-Width Neural Networks

no code implementations13 Feb 2024 Suzanna Parkinson, Greg Ongie, Rebecca Willett, Ohad Shamir, Nathan Srebro

We also show that a similar statement in the reverse direction is not possible: any function learnable with polynomial sample complexity by a norm-controlled depth-2 ReLU network with infinite width is also learnable with polynomial sample complexity by a norm-controlled depth-3 ReLU network.

Rotation-Invariant Random Features Provide a Strong Baseline for Machine Learning on 3D Point Clouds

1 code implementation27 Jul 2023 Owen Melia, Eric Jonas, Rebecca Willett

Specifically, we extend the random features method of Rahimi & Recht 2007 by deriving a version that is invariant to three-dimensional rotations and showing that it is fast to evaluate on point cloud data.

3D Shape Classification Inductive Bias +2

Integrating Uncertainty Awareness into Conformalized Quantile Regression

1 code implementation14 Jun 2023 Raphael Rossellini, Rina Foygel Barber, Rebecca Willett

The reason is that the prediction intervals of CQR do not distinguish between two forms of uncertainty: first, the variability of the conditional distribution of $Y$ given $X$ (i. e., aleatoric uncertainty), and second, our uncertainty in estimating this conditional distribution (i. e., epistemic uncertainty).

Prediction Intervals regression

Training neural operators to preserve invariant measures of chaotic attractors

1 code implementation NeurIPS 2023 Ruoxi Jiang, Peter Y. Lu, Elena Orlova, Rebecca Willett

In this paper, we propose an alternative framework designed to preserve invariant measures of chaotic attractors that characterize the time-invariant statistical properties of the dynamics.

Contrastive Learning

Deep Stochastic Mechanics

2 code implementations31 May 2023 Elena Orlova, Aleksei Ustimenko, Ruoxi Jiang, Peter Y. Lu, Rebecca Willett

This paper introduces a novel deep-learning-based approach for numerical simulation of a time-evolving Schr\"odinger equation inspired by stochastic mechanics and generative diffusion models.

Linear Neural Network Layers Promote Learning Single- and Multiple-Index Models

no code implementations24 May 2023 Suzanna Parkinson, Greg Ongie, Rebecca Willett

This paper explores the implicit bias of overparameterized neural networks of depth greater than two layers.

Bagging Provides Assumption-free Stability

1 code implementation30 Jan 2023 Jake A. Soloff, Rina Foygel Barber, Rebecca Willett

Bagging is an important technique for stabilizing machine learning models.

Reduced-Order Autodifferentiable Ensemble Kalman Filters

1 code implementation27 Jan 2023 Yuming Chen, Daniel Sanz-Alonso, Rebecca Willett

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system.


Beyond Ensemble Averages: Leveraging Climate Model Ensembles for Subseasonal Forecasting

1 code implementation29 Nov 2022 Elena Orlova, Haokun Liu, Raphael Rossellini, Benjamin A. Cash, Rebecca Willett

Producing high-quality forecasts of key climate variables, such as temperature and precipitation, on subseasonal time scales has long been a gap in operational forecasting.

Feature Importance regression

Embed and Emulate: Learning to estimate parameters of dynamical systems with uncertainty quantification

1 code implementation3 Nov 2022 Ruoxi Jiang, Rebecca Willett

This paper explores learning emulators for parameter estimation with uncertainty estimation of high-dimensional dynamical systems.

Contrastive Learning Time Series +2

NURD: Negative-Unlabeled Learning for Online Datacenter Straggler Prediction

1 code implementation16 Mar 2022 Yi Ding, Avinash Rao, Hyebin Song, Rebecca Willett, Henry Hoffmann

To predict stragglers accurately and early without labeled positive examples or assumptions on latency distributions, this paper presents NURD, a novel Negative-Unlabeled learning approach with Reweighting and Distribution-compensation that only trains on negative and unlabeled streaming data.

The Role of Linear Layers in Nonlinear Interpolating Networks

no code implementations2 Feb 2022 Greg Ongie, Rebecca Willett

This paper explores the implicit bias of overparameterized neural networks of depth greater than two layers.

Adaptive Differentially Private Empirical Risk Minimization

no code implementations14 Oct 2021 Xiaoxia Wu, Lingxiao Wang, Irina Cristali, Quanquan Gu, Rebecca Willett

We propose an adaptive (stochastic) gradient perturbation method for differentially private empirical risk minimization.

Auto-differentiable Ensemble Kalman Filters

1 code implementation16 Jul 2021 Yuming Chen, Daniel Sanz-Alonso, Rebecca Willett

Data assimilation is concerned with sequentially estimating a temporally-evolving state.

BIG-bench Machine Learning

Pure Exploration in Kernel and Neural Bandits

no code implementations NeurIPS 2021 Yinglun Zhu, Dongruo Zhou, Ruoxi Jiang, Quanquan Gu, Rebecca Willett, Robert Nowak

To overcome the curse of dimensionality, we propose to adaptively embed the feature representation of each arm into a lower-dimensional space and carefully deal with the induced model misspecification.

Prediction in the presence of response-dependent missing labels

no code implementations25 Mar 2021 Hyebin Song, Garvesh Raskutti, Rebecca Willett

In a variety of settings, limitations of sensing technologies or other sampling mechanisms result in missing labels, where the likelihood of a missing label in the training set is an unknown function of the data.

Missing Labels

Data-driven Cloud Clustering via a Rotationally Invariant Autoencoder

no code implementations8 Mar 2021 Takuya Kurihana, Elisabeth Moyer, Rebecca Willett, Davis Gilton, Ian Foster

Advanced satellite-born remote sensing instruments produce high-resolution multi-spectral data for much of the globe at a daily cadence.


Deep Equilibrium Architectures for Inverse Problems in Imaging

1 code implementation16 Feb 2021 Davis Gilton, Gregory Ongie, Rebecca Willett

Recent efforts on solving inverse problems in imaging via deep neural networks use architectures inspired by a fixed number of iterations of an optimization method.

Functional Linear Regression with Mixed Predictors

1 code implementation1 Dec 2020 Daren Wang, Zifeng Zhao, Yi Yu, Rebecca Willett

We derive finite sample theoretical guarantees and show that the excess prediction risk of our estimator is minimax optimal.

Statistics Theory Methodology Statistics Theory

Model Adaptation for Inverse Problems in Imaging

no code implementations30 Nov 2020 Davis Gilton, Gregory Ongie, Rebecca Willett

Deep neural networks have been applied successfully to a wide variety of inverse problems arising in computational imaging.

Deblurring Image Reconstruction +1

Deep Learning Techniques for Inverse Problems in Imaging

no code implementations12 May 2020 Gregory Ongie, Ajil Jalal, Christopher A. Metzler, Richard G. Baraniuk, Alexandros G. Dimakis, Rebecca Willett

Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging.

Detection and Description of Change in Visual Streams

no code implementations27 Mar 2020 Davis Gilton, Ruotian Luo, Rebecca Willett, Greg Shakhnarovich

This paper presents a framework for the analysis of changes in visual streams: ordered sequences of images, possibly separated by significant time gaps.

Change Detection Representation Learning

Context-dependent self-exciting point processes: models, methods, and risk bounds in high dimensions

no code implementations16 Mar 2020 Lili Zheng, Garvesh Raskutti, Rebecca Willett, Benjamin Mark

High-dimensional autoregressive point processes model how current events trigger or inhibit future events, such as activity by one member of a social network can affect the future activity of his or her neighbors.

Point Processes Time Series +1

An Optimal Statistical and Computational Framework for Generalized Tensor Estimation

no code implementations26 Feb 2020 Rungang Han, Rebecca Willett, Anru R. Zhang

Under mild conditions on the loss function, we establish both an upper bound on statistical error and the linear rate of computational convergence through a general deterministic analysis.


A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case

no code implementations ICLR 2020 Greg Ongie, Rebecca Willett, Daniel Soudry, Nathan Srebro

In this paper, we characterize the norm required to realize a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$ as a single hidden-layer ReLU network with an unbounded number of units (infinite width), but where the Euclidean norm of the weights is bounded, including precisely characterizing which functions can be realized with finite norm.

Learning to Solve Linear Inverse Problems in Imaging with Neumann Networks

no code implementations NeurIPS Workshop Deep_Invers 2019 Greg Ongie, Davis Gilton, Rebecca Willett

Recent advances have illustrated that it is often possible to learn to solve linear inverse problems in imaging using training data that can outperform more traditional regularized least squares solutions.

Neumann Networks for Inverse Problems in Imaging

2 code implementations13 Jan 2019 Davis Gilton, Greg Ongie, Rebecca Willett

We present an end-to-end, data-driven method of solving inverse problems inspired by the Neumann series, which we call a Neumann network.


Bilinear Bandits with Low-rank Structure

no code implementations8 Jan 2019 Kwang-Sung Jun, Rebecca Willett, Stephen Wright, Robert Nowak

We introduce the bilinear bandit problem with low-rank structure in which an action takes the form of a pair of arms from two different entity types, and the reward is a bilinear function of the known feature vectors of the arms.

Estimating Network Structure from Incomplete Event Data

no code implementations7 Nov 2018 Benjamin Mark, Garvesh Raskutti, Rebecca Willett

Multivariate Bernoulli autoregressive (BAR) processes model time series of events in which the likelihood of current events is determined by the times and locations of past events.

Time Series Time Series Analysis

Tensor Methods for Nonlinear Matrix Completion

no code implementations26 Apr 2018 Greg Ongie, Daniel Pimentel-Alarcón, Laura Balzano, Rebecca Willett, Robert D. Nowak

This approach will succeed in many cases where traditional LRMC is guaranteed to fail because the data are low-rank in the tensorized representation but not in the original representation.

Low-Rank Matrix Completion

Graph-based regularization for regression problems with alignment and highly-correlated designs

no code implementations20 Mar 2018 Yuan Li, Benjamin Mark, Garvesh Raskutti, Rebecca Willett, Hyebin Song, David Neiman

This work considers a high-dimensional regression setting in which a graph governs both correlations among the covariates and the similarity among regression coefficients -- meaning there is \emph{alignment} between the covariates and regression coefficients.

Model Selection regression

Missing Data in Sparse Transition Matrix Estimation for Sub-Gaussian Vector Autoregressive Processes

no code implementations26 Feb 2018 Amin Jalali, Rebecca Willett

In such a scenario, where covariates are highly interdependent and partially missing, new theoretical challenges arise.

Time Series Time Series Analysis

Network Estimation from Point Process Data

no code implementations13 Feb 2018 Benjamin Mark, Garvesh Raskutti, Rebecca Willett

Using our general framework, we provide a number of novel theoretical guarantees for high-dimensional self-exciting point processes that reflect the role played by the underlying network structure and long-term memory.

Point Processes

Subspace Clustering via Tangent Cones

no code implementations NeurIPS 2017 Amin Jalali, Rebecca Willett

Given samples lying on any of a number of subspaces, subspace clustering is the task of grouping the samples based on the their corresponding subspaces.

Clustering Graph Clustering

Online Learning for Changing Environments using Coin Betting

no code implementations6 Nov 2017 Kwang-Sung Jun, Francesco Orabona, Stephen Wright, Rebecca Willett

A key challenge in online learning is that classical algorithms can be slow to adapt to changing environments.

Metric Learning

Subspace Clustering with Missing and Corrupted Data

no code implementations8 Jul 2017 Zachary Charles, Amin Jalali, Rebecca Willett

Given full or partial information about a collection of points that lie close to a union of several subspaces, subspace clustering refers to the process of clustering the points according to their subspace and identifying the subspaces.


Scalable Generalized Linear Bandits: Online Computation and Hashing

no code implementations NeurIPS 2017 Kwang-Sung Jun, Aniruddha Bhargava, Robert Nowak, Rebecca Willett

Second, for the case where the number $N$ of arms is very large, we propose new algorithms in which each next arm is selected via an inner product search.

Thompson Sampling

Algebraic Variety Models for High-Rank Matrix Completion

1 code implementation ICML 2017 Greg Ongie, Rebecca Willett, Robert D. Nowak, Laura Balzano

We consider a generalization of low-rank matrix completion to the case where the data belongs to an algebraic variety, i. e. each data point is a solution to a system of polynomial equations.

Clustering Low-Rank Matrix Completion +1

Improved Strongly Adaptive Online Learning using Coin Betting

no code implementations14 Oct 2016 Kwang-Sung Jun, Francesco Orabona, Rebecca Willett, Stephen Wright

This paper describes a new parameter-free online learning algorithm for changing environments.

Metric Learning

Online Data Thinning via Multi-Subspace Tracking

no code implementations12 Sep 2016 Xin Jiang, Rebecca Willett

At the heart of this proposed approach is an online anomaly detection method based on dynamic, low-rank Gaussian mixture models.

Anomaly Detection Clustering

Inference of High-dimensional Autoregressive Generalized Linear Models

no code implementations9 May 2016 Eric C. Hall, Garvesh Raskutti, Rebecca Willett

For instance, each element of an observation vector could correspond to a different node in a network, and the parameters of an autoregressive model would correspond to the impact of the network structure on the time series evolution.

Time Series Time Series Analysis +1

On Learning High Dimensional Structured Single Index Models

no code implementations13 Mar 2016 Nikhil Rao, Ravi Ganti, Laura Balzano, Rebecca Willett, Robert Nowak

Single Index Models (SIMs) are simple yet flexible semi-parametric models for machine learning, where the response variable is modeled as a monotonic function of a linear combination of features.

Vocal Bursts Intensity Prediction

Matrix Completion Under Monotonic Single Index Models

no code implementations NeurIPS 2015 Ravi Ganti, Laura Balzano, Rebecca Willett

Most recent results in matrix completion assume that the matrix under consideration is low-rank or that the columns are in a union of low-rank subspaces.

Matrix Completion

Online Markov decision processes with Kullback-Leibler control cost

no code implementations14 Jan 2014 Peng Guan, Maxim Raginsky, Rebecca Willett

This paper considers an online (real-time) control problem that involves an agent performing a discrete-time random walk over a finite state space.

Relax but stay in control: from value to algorithms for online Markov decision processes

no code implementations28 Oct 2013 Peng Guan, Maxim Raginsky, Rebecca Willett

Online learning algorithms are designed to perform in non-stationary environments, but generally there is no notion of a dynamic state to model constraints on current and future actions as a function of past actions.

Poisson noise reduction with non-local PCA

no code implementations2 Jun 2012 Joseph Salmon, Zachary Harmany, Charles-Alban Deledalle, Rebecca Willett

Photon-limited imaging arises when the number of photons collected by a sensor array is small relative to the number of detector elements.

Astronomy Denoising +1

Near-minimax recursive density estimation on the binary hypercube

no code implementations NeurIPS 2008 Maxim Raginsky, Svetlana Lazebnik, Rebecca Willett, Jorge Silva

This paper describes a recursive estimation procedure for multivariate binary densities using orthogonal expansions.

Density Estimation

Cannot find the paper you are looking for? You can Submit a new open access paper.