Neural Partial Differential Equations
Neural ordinary differential equations (neural ODEs) introduced an approach to approximate a neural network as a system of ODEs after considering its layer as a continuous variable and discretizing its hidden dimension. While having a good characteristic that their required numbers of parameters are typically much smaller than those of conventional neural networks, neural ODEs are known to be numerically unstable and slow in solving their integral problems, resulting in the error and procrastination of the forward-pass computation. In this work, we present a novel partial differential equation (PDE)-based approach that removes the necessity of solving integral problems and considers both the layer and the hidden dimension as continuous variables. Owing to the recent advancement of learning PDEs, the presented novel concept, called \emph{neural partial differential equations} (neural PDEs), can be implemented. Our method shows comparable (or better) accuracies in much shorter forward-pass inference time for various tasks in comparison with neural ODEs.
PDF Abstract