Addtional Rebuttal (Part 2/2)
# Response to Reviewer `ufck` (2/2)
> **[Q1] How does PSL perform under conditions of extreme data sparsity? Have the authors conducted experiments to evaluate the effectiveness of PSL in such scenarios, and what mitigation strategies, if any, are suggested?**
**[Response]** Thank you for raising this concern. Note that PSL is theoretically superior to SL and it does not rely on any assumptions on dataset. Empirically, we have conducted extensive experiments on six datasets with varying sparsity (cf. Tab.B.1). The data density ranges from the highest in Amazon-Electronic (0.00208) to the lowest in Amazon-Book (0.00026), with a nearly tenfold difference. We observe that PSL consistently achieved SOTA performance with varying sparsity.
To mitigate the impact of extreme data sparsity, we believe leveraging some cold-start recommendation strategies could be useful like the work [a1, a2]. These strategies can be seamlessly integrated with PSL, requiring only the substitution of their loss functions (e.g., BPR, SL) with PSL.
**[References]**
[a1] Contrastive Learning for Cold-Start Recommendation, MM'21
[a2] A heterogeneous graph neural model for cold-start recommendation, SIGIR'21
> **[Q2] Generally, contrastive loss is sensitive to noise (false negative instances). Is it feasible to apply PSL to general contrastive learning?**
**[Response]** Insightful suggestions! PSL has the potential to be adapted as a loss function for contrastive learning (CL). In the context of CL, for a sample $z \sim X$, where the probability distributions of positive and negative samples $z_i^+, z_j^-$ are $P_z^+, P_z^-$, respectively, PSL can be defined as:
$
\mathcal{L}_{\mathrm{PSL}}(z) = \mathbb{E} _ {i \sim P_z^+} \left[\log\mathbb{E} _ {j \sim P_z^-}\left[\sigma(f(z, z_j^-)-f(z, z_j^+))^{1/\tau}\right]\right]
$
When $\sigma = \exp$ , PSL degenerates into InfoNCE. We can expect PSL equipped with relu or tanh activation functions, could mitigate the noise sensitivity by avoiding gradient explosion (cf. Fig.1(b)). We plan to further explore this promising topic in future research.
> **[Q3] In the experiments, PSL-ReLU consistently achieves sound performance across all settings. Can authors provide any rudimentary insights or intuitive understandings for the selection of activation function for broader application?**
**[Response]** In practical, we recommend using PSL-relu directly, as it achieve superior performance in most cases. Moreover, the ReLU function is simpler than the others, making it potentially easier and more efficient to optimize.
The reason for the effectiveness of PSL-relu could be explained as follows: as illustrated in Fig.1(a), on the negative half-axis, ReLU is closer to the ideal $\delta$ compared to the others. Note that $d_{uij} = f(u, j) - f(u, i) $ and in many cases the negative score $f(u, j)$ could be smaller than the positive score $f(u, i)$, indicating the $d_{uij}$ are more likely to be located at the negative half-axis. Therefore, PSL-relu acts as a *de facto* closer surrogate to DCG than others in practice, leading to better recommendation accuracy.
> **[Q4] The introduction of new activation functions in PSL may alter the training dynamics of the backbone models. Apart from performance metrics, are there differences in the objective function values or convergence rates between PSL and SL?**
**[Response]** Thanks for your question. As stated in the Response to [W2], PSL can in fact enhance the training stability by moderating the gradient distribution. We have provided the loss curves of SL and PSL in Figure R1 (in Author Rebuttal), which show that the training process of PSL is stable and converges fast.