E-FCNN for tiny facial expression recognition

As a hot issue in recent years, facial expression recognition(FER) has been widely applied in many fields, but it still faces great challenges in tiny facial expression recognition. Currently, most of the FER networks only consider images of ideal sizes. Their recognition accuracy would significantly decrease as the image resolution decreases. However, images captured by surveillance cameras often contain tiny faces with low-resolution. This paper proposes an edge-aware feedback convolutional neural network(E-FCNN) for tiny FER, which associates image super-resolution and facial expression recognition together. To effectively leverage the texture information of faces, we design a novel three-stream super-resolution network, which is embedded with an edge-enhancement block as one branch. The other two branches are the up-sampling branch and SR(Super-Resolution) primary branch. Specifically, visual features are extracted from tiny images based on a hierarchical strategy, and then put into a feedback block with fused results of the three branches. Experiments are performed on down-sampled images in four facial expression datasets: CK+, FER2013, BU-3DFE, RAF-DB. The results demonstrate the favorable performance of our network.