Rep the Set: Neural Networks for Learning Set Representations

In several domains, data objects can be decomposed into sets of simpler objects. It is then natural to represent each object as the set of its components or parts. Many conventional machine learning algorithms are unable to process this kind of representations, since sets may vary in cardinality and elements lack a meaningful ordering. In this paper, we present a new neural network architecture, called RepSet, that can handle examples that are represented as sets of vectors. The proposed model computes the correspondences between an input set and some hidden sets by solving a series of network flow problems. This representation is then fed to a standard neural network architecture to produce the output. The architecture allows end-to-end gradient-based learning. We demonstrate RepSet on classification tasks, including text categorization, and graph classification, and we show that the proposed neural network achieves performance better or comparable to state-of-the-art algorithms.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Text Classification 20NEWS ApproxRepSet Accuracy 76.18 # 13
Document Classification Amazon ApproxRepSet Accuracy 94.31 # 1
Document Classification BBCSport ApproxRepSet Accuracy 95.73 # 3
Document Classification Classic ApproxRepSet Accuracy 96.24 # 2
Graph Classification IMDb-B ApproxRepSet Accuracy 71.46% # 27
Graph Classification IMDb-M ApproxRepSet Accuracy 48.92% # 24
Graph Classification MUTAG ApproxRepSet Accuracy 86.33% # 44
Text Classification Ohsumed ApproxRepSet Accuracy 64.06 # 9
Graph Classification PROTEINS ApproxRepSet Accuracy 70.74% # 74
Document Classification Recipe ApproxRepSet Accuracy 59.06 # 1
Graph Classification REDDIT-B ApproxRepSet Accuracy 80.3 # 10
Document Classification Reuters-21578 ApproxRepSet Accuracy 97.17 # 2
Document Classification Twitter ApproxRepSet Accuracy 72.6 # 1

Methods


No methods listed for this paper. Add relevant methods here