Surprisal-Driven Zoneout

We propose a novel method of regularization for recurrent neural networks called suprisal-driven zoneout. In this method, states zoneout (maintain their previous value rather than updating), when the suprisal (discrepancy between the last state's prediction and target) is small... (read more)

Results in Papers With Code
(↓ scroll down to see all results)