Search Results for author:
Found 14 papers, 1 papers with code
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism
We evaluate the 1:2 and 2:4 sparsity under different configurations and achieve 1. 27~ 1. 89x speedups over the full-attention mechanism.
Transformer Acceleration with Dynamic Sparse Attention
We demonstrate that the sparse patterns are dynamic, depending on input sequences.
DFSSATTEN: Dynamic Fine-grained Structured Sparse Attention Mechanism
Transformers are becoming mainstream solutions for various tasks like NLP and Computer vision.
H2Learn: High-Efficiency Learning Accelerator for High-Accuracy Spiking Neural Networks
Although spiking neural networks (SNNs) take benefits from the bio-plausible neural modeling, the low accuracy under the common local synaptic plasticity learning rules limits their application in many practical tasks.
SAGA with Arbitrary Sampling
We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models.
Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling
Accelerated coordinate descent is widely used in optimization due to its cheap per-iteration cost and scalability to large-scale problems.
Quartz: Randomized Dual Coordinate Ascent with Arbitrary Sampling
We study the problem of minimizing the average of a large number of smooth convex functions penalized with a strongly convex regularizer.
Stochastic Dual Coordinate Ascent with Adaptive Probabilities
This paper introduces AdaSDCA: an adaptive variant of stochastic dual coordinate ascent (SDCA) for solving the regularized empirical risk minimization problems.
SDNA: Stochastic Dual Newton Ascent for Empirical Risk Minimization
We propose a new algorithm for minimizing regularized empirical loss: Stochastic Dual Newton Ascent (SDNA).
Coordinate Descent with Arbitrary Sampling II: Expected Separable Overapproximation
The design and complexity analysis of randomized coordinate descent methods, and in particular of variants which update a random subset (sampling) of coordinates in each iteration, depends on the notion of expected separable overapproximation (ESO).
Coordinate Descent with Arbitrary Sampling I: Algorithms and Complexity
ALPHA is a remarkably flexible algorithm: in special cases, it reduces to deterministic and randomized methods such as gradient descent, coordinate descent, parallel coordinate descent and distributed coordinate descent -- both in nonaccelerated and accelerated variants.
Randomized Dual Coordinate Ascent with Arbitrary Sampling
The distributed variant of Quartz is the first distributed SDCA-like method with an analysis for non-separable data.
Fast Distributed Coordinate Descent for Non-Strongly Convex Losses
We propose an efficient distributed randomized coordinate descent method for minimizing regularized non-strongly convex loss functions.
Squaring-Up Method In the Presence of Transmission Zeros
This paper presents a method to square up a generic MIMO system that already possesses transmission zeros.
Optimization and Control
