Logical Activation Functions: Logit-space equivalents of Probabilistic Boolean Operators

no code implementations • 22 Oct 2021 • Scott C. Lowe, Robert Earle, Jason d'Eon, Thomas Trappenberg, Sageev Oore

Consequently, we construct efficient approximations named $\text{AND}_\text{AIL}$ (the AND operator Approximate for Independent Logits), $\text{OR}_\text{AIL}$, and $\text{XNOR}_\text{AIL}$, which utilize only comparison and addition operations, have well-behaved gradients, and can be deployed as activation functions in neural networks.

Compositional Zero-Shot Learning Image Classification +1