Search Results for author:
Found 6 papers, 2 papers with code
Wasserstein Wormhole: Scalable Optimal Transport Distance with Transformers
Along with an encoder that maps distributions to embeddings, Wasserstein Wormhole includes a decoder that maps embeddings back to distributions, allowing for operations in the embedding space to generalize to OT spaces, such as Wasserstein barycenter estimation and OT interpolation.
Gradient Estimation for Binary Latent Variables via Gradient Variance Clipping
Gradient estimation is often necessary for fitting generative models with discrete latent variables, in contexts such as reinforcement learning and variational autoencoder (VAE) training.
The Ramanujan Machine: Automatically Generated Conjectures on Fundamental Constants
Fundamental mathematical constants like $e$ and $\pi$ are ubiquitous in diverse fields of science, from abstract mathematics to physics, biology and chemistry.
The effectiveness of layer-by-layer training using the information bottleneck principle
The recently proposed information bottleneck (IB) theory of deep nets suggests that during training, each layer attempts to maximize its mutual information (MI) with the target labels (so as to allow good prediction accuracy), while minimizing its MI with the input (leading to effective compression and thus good generalization).
DON’T JUDGE A BOOK BY ITS COVER - ON THE DYNAMICS OF RECURRENT NEURAL NETWORKS
To be effective in sequential data processing, Recurrent Neural Networks (RNNs) are required to keep track of past events by creating memories.
Understanding and Controlling Memory in Recurrent Neural Networks
Finally, we propose a novel regularization technique that is based on the relation between hidden state speeds and memory longevity.
