Search Results for author:
Found 7 papers, 3 papers with code
Cost-efficient Gaussian Tensor Network Embeddings for Tensor-structured Inputs
We provide a systematic way to design tensor network embeddings consisting of Gaussian random tensors, such that for inputs with more general tensor network structures, both the sketch size (row size of $S$) and the sketching computational cost are low.
LEAP: Learnable Pruning for Transformer-based Models
Moreover, in order to reduce hyperparameter tuning, a novel adaptive regularization coefficient is deployed to control the regularization penalty adaptively.
Fast and Accurate Randomized Algorithms for Low-rank Tensor Decompositions
Experimental results show that this new ALS algorithm, combined with a new initialization scheme based on randomized range finder, yields up to $22. 0\%$ relative decomposition residual improvement compared to the state-of-the-art sketched randomized algorithm for Tucker decomposition of various synthetic and real datasets.
AutoHOOT: Automatic High-Order Optimization for Tensors
High-order optimization methods, including Newton's method and its variants as well as alternating minimization methods, dominate the optimization algorithms for tensor decompositions and tensor networks.
Mathematical Software Numerical Analysis Numerical Analysis
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
In particular, we propose a new group-wise quantization scheme, and we use a Hessian based mix-precision method to compress the model further.
Inefficiency of K-FAC for Large Batch Size Training
In stochastic optimization, using large batch sizes during training can leverage parallel resources to produce faster wall-clock training times per training epoch.
Accelerating Alternating Least Squares for Tensor Decomposition by Pairwise Perturbation
The alternating least squares algorithm for CP and Tucker decomposition is dominated in cost by the tensor contractions necessary to set up the quadratic optimization subproblems.
Numerical Analysis Numerical Analysis
