Further Response about the Non-negativity Constraint
Dear Review fFnJ,
We would like to provide further details and discussions about the non-negativity constraint. In this response, we will first elaborate on how we integrated this constraint into our model. Subsequently, we will present findings to demonstrate that the absence of this constraint has not influenced our experimental results.
With the incorporation of the non-negativity constraint, the optimization problem is described as:
$$
\begin{aligned}
& \underset{\mathbf{B}}{\arg \min }\|\mathbf{I}-\mathbf{B}\|_F^2+\lambda \operatorname{tr}\left(\mathbf{B}^{\top} \tilde{\mathbf{L}} \mathbf{B}\right) \\\\
& \text { s.t. } \quad \mathbf{B} \mathbf{T} \mathbf{1}_n=c \mathbf{1}_n, \quad\mathbf{B}\ge0
\end{aligned}
$$
To tackle this problem, we employed the Lagrange Multiplier method coupled with projected gradient descent. Specifically, the iterations for $\mathbf{B}$ are:
$$
\begin{aligned}
& \mathbf{B}^{k+1}=\underset{\mathbf{B}}{\arg \min } L_\rho\left(\mathbf{B}^k, \mathbf{y}^k\right) \\\\
& \mathbf{B}^{k+1} = max(0, \mathbf{B}^{k+1}) \\\\
& \mathbf{y}^{k+1}=\mathbf{y}^k+\rho\left(\mathbf{B}^{k+1} \mathbf{T} \mathbf{1}_n-c \mathbf{1}_n\right),
\end{aligned}
$$
where $L_\rho(\mathbf{B}, \mathbf{y})=\|\mathbf{I}-\mathbf{B}\|_F^2+\lambda \operatorname{tr}\left(\mathbf{B}^{\top} \tilde{\mathbf{L}} \mathbf{B}\right)+\mathbf{y}^{\top}\left(\mathbf{B} \mathbf{T} \mathbf{1}_n-c \mathbf{1}_n\right)+\frac{\rho}{2}\left\|\mathbf{B} \mathbf{T} \mathbf{1}_n-c \mathbf{1}_n\right\|_2^2$ is the augmented Lagrange function.
From previous discussions, we've made several observations:
1. Only when the hyperparameter **c** is extremely small does $\mathbf{B}$ contain a few minor negative entries.
2. As evidenced in **A4**, introducing the non-negativity constraint has a negligible impact on performance under the same hyperparameter settings.
3. In our experiments, the optimal $\mathbf{B}$ was found to be non-negative under the best hyperparameters.
We further test whether the non-negative $\textbf{B}$ would be changed after applying the non-negativity constrain. Here is the finding:
4. For non-negative $\textbf{B}$ derived without the non-negativity constraint, reintroducing the constraint didn't alter $\textbf{B}$, ensuring consistent performance.
These findings indicate that the absence of the non-negativity constraint has not substantially influenced our experimental results.
Of course, the non-negativity constrain is essential, and we will add it into our revision. Thanks again.
We hope this provides clarity on the matter. If you have any further questions, we are happy to discuss.
Best,
All authors.