Recurrently Controlling a Recurrent Network with Recurrent Networks Controlled by More Recurrent Networks
This paper explores an intriguing idea of recursively parameterizing recurrent nets. Simply speaking, this refers to recurrently controlling a recurrent network with recurrent networks controlled by recurrent networks. The proposed architecture recursively parameterizes its gating functions whereby gating mechanisms of X-RNN are controlled by instances of itself, which are repeatedly called in a recursive fashion. We postulate that our proposed inductive bias provides modeling benefits pertaining to learning with inherently hierarchically-structured sequence data. To this end, we conduct extensive experiments on recursive logic tasks (sorting, tree traversal, logical inference), sequential pixel-by-pixel classification, semantic parsing, code generation, machine translation and polyphonic music modeling, demonstrating the widespread utility of the proposed approach, i.e., achieving optimistic and competitive results on all tasks.
PDF Abstract
