Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets

1 code implementation • 23 Jun 2022 • Lily H. Zhang, Veronica Tozzo, John M. Higgins, Rajesh Ranganath

However, we show that existing permutation invariant architectures, Deep Sets and Set Transformer, can suffer from vanishing or exploding gradients when they are deep.