Invariant Feature Learning by Attribute Perception Matching

An adversarial feature learning (AFL) is a powerful framework to learn representations invariant to a nuisance attribute, which uses an adversarial game between a feature extractor and a categorical attribute classifier. It theoretically sounds in term of it maximize conditional entropy between attribute and representation. However, as shown in this paper, the AFL often causes unstable behavior that slows down the convergence. We propose an {\em attribute perception matching} as an alternative approach, based on the reformulation of conditional entropy maximization as {\em pair-wise distribution matching}. Although the naive approach for realizing the pair-wise distribution matching requires the significantly large number of parameters, the proposed method requires the same number of parameters with AFL but has a better convergence property. Experiments on both toy and real-world dataset prove that our proposed method converges to better invariant representation significantly faster than AFL.