Search Results for author:
Found 15 papers, 3 papers with code
Adaptive Regret for Bandits Made Possible: Two Queries Suffice
In this paper, we give query and regret optimal bandit algorithms under the strict notion of strongly adaptive regret, which measures the maximum regret over any contiguous interval $I$.
Getting aligned on representational alignment
Finally, we lay out open problems in representational alignment where progress can benefit all three of these fields.
Optimal Scalarizations for Sublinear Hypervolume Regret
Scalarization is a general technique that can be deployed in any multiobjective setting to reduce multiple objectives into one, such as recently in RLHF for training reward models that align human preferences.
Set Learning for Accurate and Calibrated Models
Model overconfidence and poor calibration are common in machine learning and difficult to account for when applying standard empirical risk minimization.
Optimal Query Complexities for Dynamic Trace Estimation
Specifically, for any $m$ matrices $A_1,..., A_m$ with consecutive differences bounded in Schatten-$1$ norm by $\alpha$, we provide a novel binary tree summation procedure that simultaneously estimates all $m$ traces up to $\epsilon$ error with $\delta$ failure probability with an optimal query complexity of $\widetilde{O}\left(m \alpha\sqrt{\log(1/\delta)}/\epsilon + m\log(1/\delta)\right)$, improving the dependence on both $\alpha$ and $\delta$ from Dharangutte and Musco (NeurIPS, 2021).
Towards Learning Universal Hyperparameter Optimizers with Transformers
Meta-learning hyperparameter optimization (HPO) algorithms from prior experiments is a promising approach to improve optimization efficiency over objective functions from a similar distribution.
One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks
Can deep learning solve multiple tasks simultaneously, even when they are unrelated and very different?
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
In this paper, we approach the problem of optimizing blackbox functions over large hybrid search spaces consisting of both combinatorial and continuous parameters.
Joint Descent: Training and Tuning Simultaneously
Typically in machine learning, training and tuning are done in an alternating manner: for a fixed set of hyperparameters $y$, we apply gradient descent to our objective $f(x, y)$ over trainable variables $x$ until convergence; then, we apply a tuning step over $y$ to find another promising setting of hyperparameters.
Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimization
Single-objective black box optimization (also known as zeroth-order optimization) is the process of minimizing a scalar objective $f(x)$, given evaluations at adaptively chosen inputs $x$.
Learning the gravitational force law and other analytic functions
We present experimental evidence that the many-body gravitational force function is easier to learn with ReLU networks as compared to networks with exponential activations.
Regularized Weighted Low Rank Approximation
The classical low rank approximation problem is to find a rank $k$ matrix $UV$ (where $U$ has $k$ columns and $V$ has $k$ rows) that minimizes the Frobenius norm of $A - UV$.
Gradientless Descent: High-Dimensional Zeroth-Order Optimization
Zeroth-order optimization is the process of minimizing an objective $f(x)$, given oracle access to evaluations at adaptively chosen inputs $x$.
Solving Empirical Risk Minimization in the Current Matrix Multiplication Time
Our result generalizes the very recent result of solving linear programs in the current matrix multiplication time [Cohen, Lee, Song'19] to a more broad class of problems.
Convergence Results for Neural Networks via Electrodynamics
Iterating, we show that gradient descent can be used to learn the entire network one node at a time.
