Product-based Neural Networks for User Response Prediction

1 Nov 2016  ·  Yanru Qu, Han Cai, Kan Ren, Wei-Nan Zhang, Yong Yu, Ying Wen, Jun Wang ·

Predicting user responses, such as clicks and conversions, is of great importance and has found its usage in many Web applications including recommender systems, web search and online advertising. The data in those applications is mostly categorical and contains multiple fields; a typical representation is to transform it into a high-dimensional sparse binary feature representation via one-hot encoding... Facing with the extreme sparsity, traditional models may limit their capacity of mining shallow patterns from the data, i.e. low-order feature combinations. Deep models like deep neural networks, on the other hand, cannot be directly applied for the high-dimensional input because of the huge feature space. In this paper, we propose a Product-based Neural Networks (PNN) with an embedding layer to learn a distributed representation of the categorical data, a product layer to capture interactive patterns between inter-field categories, and further fully connected layers to explore high-order feature interactions. Our experimental results on two large-scale real-world ad click datasets demonstrate that PNNs consistently outperform the state-of-the-art models on various metrics. read more

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Click-Through Rate Prediction Amazon PNN AUC 0.8679 # 4
Click-Through Rate Prediction Bing News PNN AUC 0.8321 # 4
Log Loss 0.2775 # 4
Click-Through Rate Prediction Company* PNN* AUC 0.8672 # 4
Log Loss 0.02636 # 4
Click-Through Rate Prediction Company* IPNN AUC 0.8664 # 5
Log Loss 0.02637 # 5
Click-Through Rate Prediction Company* OPNN AUC 0.8658 # 7
Log Loss 0.02641 # 7
Click-Through Rate Prediction Criteo IPNN AUC 0.7972 # 17
Log Loss 0.45323 # 11
Click-Through Rate Prediction Criteo OPNN AUC 0.7982 # 15
Log Loss 0.45256 # 10
Click-Through Rate Prediction Criteo PNN* AUC 0.7987 # 14
Log Loss 0.45214 # 9
Click-Through Rate Prediction Dianping PNN AUC 0.8445 # 3
Log Loss 0.3424 # 4
Click-Through Rate Prediction iPinYou PNN* AUC 0.7661 # 3
Click-Through Rate Prediction iPinYou IPNN AUC 0.7914 # 2
Click-Through Rate Prediction iPinYou OPNN AUC 0.8174 # 1
Click-Through Rate Prediction MovieLens 20M PNN AUC 0.7321 # 6

Methods


No methods listed for this paper. Add relevant methods here