Search Results for author:
Found 6 papers, 2 papers with code
Automatic Differentiation: Theory and Practice
We present the classical coordinate-free formalism for forward and backward mode ad in the real and complex setting.
Geometric Optimisation on Manifolds with Applications to Deep Learning
We design and implement a Python library to help the non-expert using all these powerful tools in a way that is efficient, extensible, and simple to incorporate into the workflow of the data scientist, practitioner, and applied researcher.
Adaptive and Momentum Methods on Manifolds Through Trivializations
We introduce a framework to generalize adaptive and momentum methods to arbitrary manifolds by noting that for every differentiable manifold, there exists a radially convex open set that covers almost all the manifold.
Curvature-Dependant Global Convergence Rates for Optimization on Manifolds of Bounded Geometry
We give curvature-dependant convergence rates for the optimization of weakly convex functions defined on a manifold of 1-bounded geometry via Riemannian gradient descent and via the dynamic trivialization algorithm.
Trivializations for Gradient-Based Optimization on Manifolds
We prove conditions under which a trivialization is sound in the context of gradient-based optimization and we show how two large families of trivializations have overall favorable properties, but also suffer from a performance issue.
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group
We demonstrate how our method constitutes a more robust approach to optimization with orthogonal constraints, showing faster, accurate, and more stable convergence in several tasks designed to test RNNs.
