1 code implementation • 24 Mar 2022 • Yizhe Xu, Steve Yadlowsky
However, while many methods exist for evaluating the calibration of prediction and classification models, formal approaches to assess the calibration of HTE models are limited to the calibration slope.
no code implementations • 15 Nov 2021 • Steve Yadlowsky, Scott Fleming, Nigam Shah, Emma Brunskill, Stefan Wager
On the other hand, in a large marketing trial, we find robust evidence of heterogeneity in the treatment effects of some digital advertising campaigns and demonstrate how RATEs can be used to compare targeting rules that prioritize estimated risk vs. those that prioritize estimated treatment benefit.
1 code implementation • ICLR 2022 • Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D'Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, Ian Tenney, Ellie Pavlick
Experiments with pre-trained models such as BERT are often based on a single checkpoint.
no code implementations • NeurIPS 2021 • Victor Veitch, Alexander D'Amour, Steve Yadlowsky, Jacob Eisenstein
We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions.
no code implementations • NeurIPS 2021 • Victor Veitch, Alexander D'Amour, Steve Yadlowsky, Jacob Eisenstein
We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions.
1 code implementation • NeurIPS 2021 • Steve Yadlowsky, Taedong Yun, Cory McLean, Alexander D'Amour
The key insight of SLOE is that the Sur and Cand\`es (2019) correction can be reparameterized in terms of the \emph{corrupted signal strength}, which is only a function of the estimated parameters $\widehat \beta$.
4 code implementations • 16 Jan 2021 • Chirag Nagpal, Steve Yadlowsky, Negar Rostamzadeh, Katherine Heller
Survival analysis is a challenging variation of regression modeling because of the presence of censoring, where the outcome measurement is only partially known, due to, for example, loss to follow up.
no code implementations • 6 Nov 2020 • Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley
Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains.
1 code implementation • NeurIPS 2020 • Hongseok Namkoong, Ramtin Keramati, Steve Yadlowsky, Emma Brunskill
We assess robustness of OPE methods under unobserved confounding by developing worst-case bounds on the performance of an evaluation policy.
no code implementations • 15 Dec 2019 • Steve Yadlowsky, Fabio Pellegrini, Federica Lionetto, Stefan Braune, Lu Tian
Motivated by the need of modeling the number of relapses in multiple sclerosis patients, where the ratio of relapse rates is a natural choice of the treatment effect, we propose to estimate the conditional average treatment effect (CATE) as the ratio of expected potential outcomes, and derive a doubly robust estimator of this CATE in a semiparametric model of treatment-covariate interactions.
1 code implementation • 11 Apr 2018 • Tatsunori B. Hashimoto, Steve Yadlowsky, John C. Duchi
We develop an algorithm for minimizing a function using $n$ batched function value measurements at each of $T$ rounds by using classifiers to identify a function's sublevel set.
no code implementations • ICML 2017 • Hongseok Namkoong, Aman Sinha, Steve Yadlowsky, John C. Duchi
Standard forms of coordinate and stochastic gradient methods do not adapt to structure in data; their good behavior under random sampling is predicated on uniformity in data.