Joint Learning of Full-structure Noise in Hierarchical Bayesian Regression Models

We consider hierarchical Bayesian (type-II maximum likelihood) regression models for observations with latent variables for source and noise, where parameters of priors for source and noise terms need to be estimated jointly from data. This problem has application in many domains in imaging including biomagnetic inverse problems. Crucial factors influencing accuracy of source estimation are not only the noise level but also its correlation structure, but existing approaches have not addressed estimation of a full-structure noise covariance matrix. Here, we focus on sparse Bayesian learning (SBL) in regression models specifically for the application of reconstruction of brain activity from electroencephalography (EEG), which can be formulated as a linear regression with independent Gaussian scale mixture priors for both the source and noise components. As a departure from the classical SBL models where across sensor observations are assumed to be independent and identically distributed, we consider Gaussian noise with full covariance structure. Using ideas from Riemannian geometry, we derive an efficient algorithm for updating both source and the noise covariance along the manifold of positive definite matrices. We verify the theoretical advantages of our model and algorithm in extensive simulations in the context of the EEG inverse problem. Our results demonstrate that the novel framework significantly improves upon state-of-the-art techniques in the real-world scenario where the noise is indeed non-diagonal and fully-structured.