Search Results for author:
Found 10 papers, 5 papers with code
G-RepsNet: A Fast and General Construction of Equivariant Networks for Arbitrary Matrix Groups
Recent work by Finzi et al. (2021) directly solves the equivariance constraint for arbitrary matrix groups to obtain equivariant MLPs (EMLPs).
Efficient Model-Agnostic Multi-Group Equivariant Networks
Inspired by the first design, we use the notion of the IS property to design a second efficient model-agnostic equivariant design for large product groups acting on a single input.
Transformers are Universal Predictors
We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense.
Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models
We also provide a novel group-theoretic definition for fairness in NLG.
Equivariant Mesh Attention Networks
Equivariance to symmetries has proven to be a powerful inductive bias in deep learning research.
Autoequivariant Network Search via Group Decomposition
We address these problems by proving a new group-theoretic result in the context of equivariant neural networks that shows that a network is equivariant to a large group if and only if it is equivariant to smaller groups from which it is constructed.
The Twelvefold Way of Non-Sequential Lossless Compression
Many information sources are not just sequences of distinguishable symbols but rather have invariances governed by alternative counting paradigms such as permutations, combinations, and partitions.
Information Theory Information Theory
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Experiments show that for low values of k and p in top-k and top-p sampling, perplexity drops significantly with generated text length, which is also correlated with excessive repetitions in the text (the boredom trap).
Succinct Source Coding of Deep Neural Networks
Deep neural networks have shown incredible performance for inference tasks in a variety of domains.
Universal and Succinct Source Coding of Deep Neural Networks
Deep neural networks have shown incredible performance for inference tasks in a variety of domains.
