Generalized Separable Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a linear dimensionality technique for nonnegative data with applications such as image analysis, text mining, audio source separation, and hyperspectral unmixing. Given a data matrix <inline-formula><tex-math notation="LaTeX">$M$</tex-math><alternatives><mml:math><mml:mi>M</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq1-2956046.gif"/></alternatives></inline-formula> and a factorization rank <inline-formula><tex-math notation="LaTeX">$r$</tex-math><alternatives><mml:math><mml:mi>r</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq2-2956046.gif"/></alternatives></inline-formula>, NMF looks for a nonnegative matrix <inline-formula><tex-math notation="LaTeX">$W$</tex-math><alternatives><mml:math><mml:mi>W</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq3-2956046.gif"/></alternatives></inline-formula> with <inline-formula><tex-math notation="LaTeX">$r$</tex-math><alternatives><mml:math><mml:mi>r</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq4-2956046.gif"/></alternatives></inline-formula> columns and a nonnegative matrix <inline-formula><tex-math notation="LaTeX">$H$</tex-math><alternatives><mml:math><mml:mi>H</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq5-2956046.gif"/></alternatives></inline-formula> with <inline-formula><tex-math notation="LaTeX">$r$</tex-math><alternatives><mml:math><mml:mi>r</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq6-2956046.gif"/></alternatives></inline-formula> rows such that <inline-formula><tex-math notation="LaTeX">$M \approx WH$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>M</mml:mi><mml:mo>≈</mml:mo><mml:mi>W</mml:mi><mml:mi>H</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="gillis-ieq7-2956046.gif"/></alternatives></inline-formula>. NMF is NP-hard to solve in general. However, it can be computed efficiently under the separability assumption which requires that the basis vectors appear as data points, that is, that there exists an index set <inline-formula><tex-math notation="LaTeX">$\mathcal {K}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">K</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq8-2956046.gif"/></alternatives></inline-formula> such that <inline-formula><tex-math notation="LaTeX">$W = M(:,\mathcal {K})$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>W</mml:mi><mml:mo>=</mml:mo><mml:mi>M</mml:mi><mml:mo>(</mml:mo><mml:mo>:</mml:mo><mml:mo>,</mml:mo><mml:mi mathvariant="script">K</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="gillis-ieq9-2956046.gif"/></alternatives></inline-formula>. In this article, we generalize the separability assumption. We only require that for each rank-one factor <inline-formula><tex-math notation="LaTeX">$W(:,k)H(k,:)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>W</mml:mi><mml:mo>(</mml:mo><mml:mo>:</mml:mo><mml:mo>,</mml:mo><mml:mi>k</mml:mi><mml:mo>)</mml:mo><mml:mi>H</mml:mi><mml:mo>(</mml:mo><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mo>:</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="gillis-ieq10-2956046.gif"/></alternatives></inline-formula> for <inline-formula><tex-math notation="LaTeX">$k=1,2,\ldots,r$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mo>...</mml:mo><mml:mo>,</mml:mo><mml:mi>r</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="gillis-ieq11-2956046.gif"/></alternatives></inline-formula>, either <inline-formula><tex-math notation="LaTeX">$W(:,k) = M(:,j)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>W</mml:mi><mml:mo>(</mml:mo><mml:mo>:</mml:mo><mml:mo>,</mml:mo><mml:mi>k</mml:mi><mml:mo>)</mml:mo><mml:mo>=</mml:mo><mml:mi>M</mml:mi><mml:mo>(</mml:mo><mml:mo>:</mml:mo><mml:mo>,</mml:mo><mml:mi>j</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="gillis-ieq12-2956046.gif"/></alternatives></inline-formula> for some <inline-formula><tex-math notation="LaTeX">$j$</tex-math><alternatives><mml:math><mml:mi>j</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq13-2956046.gif"/></alternatives></inline-formula> or <inline-formula><tex-math notation="LaTeX">$H(k,:) = M(i,:)$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>H</mml:mi><mml:mo>(</mml:mo><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mo>:</mml:mo><mml:mo>)</mml:mo><mml:mo>=</mml:mo><mml:mi>M</mml:mi><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mo>:</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="gillis-ieq14-2956046.gif"/></alternatives></inline-formula> for some <inline-formula><tex-math notation="LaTeX">$i$</tex-math><alternatives><mml:math><mml:mi>i</mml:mi></mml:math><inline-graphic xlink:href="gillis-ieq15-2956046.gif"/></alternatives></inline-formula>. We refer to the corresponding problem as generalized separable NMF (GS-NMF). We discuss some properties of GS-NMF and propose a convex optimization model which we solve using a fast gradient method. We also propose a heuristic algorithm inspired by the successive projection algorithm. To verify the effectiveness of our methods, we compare them with several state-of-the-art separable NMF and standard NMF algorithms on synthetic, document and image data sets.