Search Results for author:
Found 11 papers, 1 papers with code
Quantum Speedup for Spectral Approximation of Kronecker Products
Given its widespread application in machine learning and optimization, the Kronecker product emerges as a pivotal linear algebra operator.
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time
$A_3$ is a matrix in $\mathbb{R}^{n \times d}$, $\mathsf{A}_{j_0} \in \mathbb{R}^{n \times d^2}$ is the $j_0$-th block of $\mathsf{A}$.
GradientCoin: A Peer-to-Peer Decentralized Large Language Models
It is likely that only two types of people would be interested in setting up a practical system for it: $\bullet$ Those who prefer to use a decentralized ChatGPT-like software.
Fast Quantum Algorithm for Attention Computation
It is well-known that quantum machine has certain computational advantages compared to the classical machine.
In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick
Given matrices $A_1 \in \mathbb{R}^{n \times d}$, and $A_2 \in \mathbb{R}^{n \times d}$ and $B \in \mathbb{R}^{n \times n}$, the purpose is to solve some certain optimization problems: Normalized version $\min_{X} \| D(X)^{-1} \exp(A_1 X A_2^\top) - B \|_F^2$ and Rescaled version $\| \exp(A_1 X A_2^\top) - D(X) \cdot B \|_F^2$.
Differentially Private Attention Computation
Inspired by [Vyas, Kakade and Barak 2023], in this work, we provide a provable result for showing how to differentially private approximate the attention matrix.
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression
LLMs have shown great promise in improving the accuracy and efficiency of these tasks, and have the potential to revolutionize the field of natural language processing (NLP) in the years to come.
Solving Tensor Low Cycle Rank Approximation
For the tensor classical rank, tucker rank and train rank, it has been well studied in [Song, Woodruff, Zhong SODA 2019].
An Over-parameterized Exponential Regression
Mathematically, we define the neural function $F: \mathbb{R}^{d \times m} \times \mathbb{R}^d \rightarrow \mathbb{R}$ using an exponential activation function.
A Sublinear Adversarial Training Algorithm
For a neural network of width $m$, $n$ input training data in $d$ dimension, it takes $\Omega(mnd)$ time cost per training iteration for the forward and backward computation.
Optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms
Graph neural networks (GNN) have achieved state-of-the-art performance on various industrial tasks.
