no code implementations • 7 Feb 2020 • Gil I. Shamir
We derive lower bounds with logarithmic regret under $L_1$, $L_2$, and $L_\infty$ constraints on the parameter values.
no code implementations • 19 Oct 2020 • Gil I. Shamir, Lorenzo Coviello
Deep networks have been revolutionary in improving performance of machine learning and artificial intelligence systems.
1 code implementation • 20 Oct 2020 • Gil I. Shamir, Dong Lin, Lorenzo Coviello
We propose a new family of activations; Smooth ReLU (\emph{SmeLU}), designed to give such better tradeoffs, while also keeping the mathematical expression simple, and thus implementation cheap.
no code implementations • 28 Jan 2021 • Gil I. Shamir, Wojciech Szpankowski
Various approximations that, for huge sparse feature sets, diminish the theoretical advantages, must be used.
no code implementations • 21 Feb 2021 • Robert R. Snapp, Gil I. Shamir
We show that even with a single nonlinearity and for very simple data and models, irreproducibility occurs.
no code implementations • 9 Feb 2022 • Kwangjun Ahn, Prateek Jain, Ziwei Ji, Satyen Kale, Praneeth Netrapalli, Gil I. Shamir
We initiate a formal study of reproducibility in optimization.
2 code implementations • 14 Feb 2022 • Gil I. Shamir, Dong Lin
We describe a novel family of smooth activations; Smooth ReLU (SmeLU), designed to improve reproducibility with mathematical simplicity, with potentially cheaper implementation.
no code implementations • 12 Sep 2022 • Rohan Anil, Sandra Gadanho, Da Huang, Nijith Jacob, Zhuoshu Li, Dong Lin, Todd Phillips, Cristina Pop, Kevin Regan, Gil I. Shamir, Rakesh Shivanna, Qiqi Yan
For industrial-scale advertising systems, prediction of ad click-through rate (CTR) is a central problem.