Variational Discriminant Analysis with Variable Selection

A Bayesian method that seamlessly fuses classification via discriminant analysis and hypothesis testing is developed. Building upon the original discriminant analysis classifier, modelling components are added to identify discriminative variables. A combination of cake priors and a novel form of variational Bayes we call reverse collapsed variational Bayes gives rise to variable selection that can be directly posed as a multiple hypothesis testing approach using likelihood ratio statistics. Some theoretical arguments are presented showing that Chernoff-consistency (asymptotically zero type I and type II error) is maintained across all hypotheses. We apply our method on some publicly available genomic datasets and show that our method performs well in practice. An R package VaDA has also been made available on Github.