Linear Regression on a Set of Selected Templates from a Pool of Randomly Generated Templates

We study linear regression for two datasets. For the MNIST dataset we do so using max convolutions, whose parameters are generated directly from training images for the digit recognition problem, hence we call them max convolution templates. From a large pool of randomly generated convolutional templates, we select by iterative process the ones which improve defined linear regression minimization problem the most. With these templates, we use linear and logistic regression and achieve high accuracy, comparable with deep neural networks. We explain why, in a production environment, using this approach has advantages over the use of deep neural networks. On a second dataset ‘Adult Data Set’ of income predictions, we show a similar convolution type approach for generating a pool of random templates and show that the same template selection process and linear regression can be used as for the MNIST dataset.