Scalable Gradients for Stochastic Differential Equations

5 Jan 2020  ·  Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, David Duvenaud ·

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers... Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge. In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations. We use our method to fit stochastic dynamics defined by neural networks, achieving competitive performance on a 50-dimensional motion capture dataset. read more

PDF Abstract

Datasets


  Add Datasets introduced or used in this paper

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Video Prediction CMU Mocap-2 Latent SDE Test Error 4.03 # 1
Video Prediction CMU Mocap-2 Latent ODE Test Error 5.98 # 2

Methods