Search Results for author:
Found 3 papers, 2 papers with code
Rethinking Optimization and Architecture for Tiny Language Models
Several design formulas are empirically proved especially effective for tiny language models, including tokenizer compression, architecture tweaking, parameter inheritance and multiple-round training.
Cascaded Algorithm-Selection and Hyper-Parameter Optimization with Extreme-Region Upper Confidence Bound Bandit
We show theoretically that the ER-UCB has a regret upper bound $O\left(K \ln n\right)$ with independent feedbacks, which is as efficient as the classical UCB bandit.
ZOOpt: Toolbox for Derivative-Free Optimization
Recent advances in derivative-free optimization allow efficient approximation of the global-optimal solutions of sophisticated functions, such as functions with many local optima, non-differentiable and non-continuous functions.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
