Sparse Bayesian Dictionary Learning with a Gaussian Hierarchical Model

We consider a dictionary learning problem aimed at designing a dictionary such that the signals admits a sparse or an approximate sparse representation over the learned dictionary. The problem finds a variety of applications including image denoising, feature extraction, etc. In this paper, we propose a new hierarchical Bayesian model for dictionary learning, in which a Gaussian-inverse Gamma hierarchical prior is used to promote the sparsity of the representation. Suitable non-informative priors are also placed on the dictionary and the noise variance such that they can be reliably estimated from the data. Based on the hierarchical model, a Gibbs sampling method is developed for Bayesian inference. The proposed method have the advantage that it does not require the knowledge of the noise variance a priori. Numerical results show that the proposed method is able to learn the dictionary with an accuracy better than existing methods.