Search Results for author:
Found 8 papers, 3 papers with code
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
In this paper, we present AquilaMoE, a cutting-edge bilingual 8*16B Mixture of Experts (MoE) language model that has 8 experts with 16 billion parameters each and is developed using an innovative training methodology called EfficientScale.
Improving the List Decoding Version of the Cyclically Equivariant Neural Decoder
In the same paper, a list decoding procedure was also introduced for two widely used classes of cyclic codes -- BCH codes and punctured Reed-Muller (RM) codes.
Cyclically Equivariant Neural Decoders for Cyclic Codes
Finally, we propose a list decoding procedure that can significantly reduce the decoding error probability for BCH codes and punctured RM codes.
Exact recovery and sharp thresholds of Stochastic Ising Block Model
We show that when $m\ge m^\ast$, one can recover the clusters from $m$ samples in $O(n)$ time as the number of vertices $n$ goes to infinity.
Optimal locally private estimation under $\ell_p$ loss for $1\le p\le 2$
In this paper, we sharpen this result by showing asymptotic optimality of the proposed scheme under the $\ell_p^p$ loss for all $1\le p\le 2.$ More precisely, we show that for any $p\in[1, 2]$ and any $k$ and $\epsilon,$ the ratio between the worst-case $\ell_p^p$ estimation loss of our scheme and the optimal value approaches $1$ as the number of samples tends to infinity.
Communication-Computation Efficient Gradient Coding
This paper develops coding techniques to reduce the running time of distributed learning tasks.
Asymptotically optimal private estimation under mean square loss
In other words, for a large number of samples the worst-case estimation loss of our scheme was shown to differ from the optimal value by at most a constant factor.
Optimal Schemes for Discrete Distribution Estimation under Locally Differential Privacy
For a given $\epsilon,$ we consider the problem of constructing optimal privatization schemes with $\epsilon$-privacy level, i. e., schemes that minimize the expected estimation loss for the worst-case distribution.
