Responce to your further concerns.
Dear Reviewer VUDN,
Thank you very much for your valuable feedback. We sincerely hope that our following response could properly address your concerns.
1. >*Softmax-based approaches should also be considered as generative methods since they directly model the probability of a target item being the next item. In that way, the distinction between softmax and diffusion is really marginal.*
We do agree that softmax directly models the probability of a target item being the next item. Meanwhile, we would like to kindly emphasize that **softmax and generative models are different due to their adherence to distinct paradigms (discriminative and generative respectively)**.
Specifically, generative models (GANs, VAEs and diffusion models, e.t.c) directly model the underlying data generation distribution by learning the map from Gaussian distribution and the underlying distribution. **In contrast, softmax is commonly employed in discriminative models, since it models the probability distribution over a set of discrete classes, with limited presence in the literature of generative models**. Besides, softmax is limited to the discrete candidate set, whereas diffusion can generate samples beyond the candidate. Therefore, we would respectfully emphasize that softmax and diffusion models are quite distinctive.
2. >*The authors argued that softmax-based approaches explicitly use negative samples, while diff4rec does not. In fact, both softmax and diffusion implicitly use negative sampling. If a diffusion model does not implicitly use negative sampling, the model will simply collapse.*
We agree that softmax implicitly uses negative sampling (we feel sorry about the potential confusion caused by that it should be 'implicitly' instead of 'explicitly' in how softmax uses unobserved interactions in the table of Point 1 in the rebuttal). Meanwhile, we would kindly emphasize that **negative sampling is not used in Diff4Rec, either explicitly or implicitly**. The primary objective of Diff4Rec is:
$L_{t-1} = \mathbb{E}_{e_n\^0, \epsilon}\left[\frac{\bar\alpha\_{t-1}}{2\tilde{\beta}\_t} ||e\_n^0 - f\_\theta(\sqrt{\bar\alpha\_t} e_n^{0} + \sqrt{1-\bar\alpha\_t}\epsilon, c\_{n-1}, t)||^2\right] + C,$
where $e_n^0$ is sampled from **only observed interactions**. We have also provided the code and training algorithm, and we hope these materials could help address your concern.
**Even if negative sampling is not used (implicitly or explicitly) in Diff4Rec, it would not collapse, which is one of the contributions of this work**. The reason can be attributed to the fundamental paradigm shift: the paradigm of Diff4Rec is learning-to-generate through diffusion, instead of traditional learning-to-classify. Specifically, Diff4Rec distinctly models the underlying generation distribution of observed interactions with the power of diffusion, which does not require negative samples. However, previous sequential recommenders adhere to the learning-to-classify paradigm and distinguish positive and negative samples, which strictly requires negative sampling.
**We do understand that recommendation has long been recognized as a discriminative task requiring negative sampling. Yet our work does show that sequential recommendation can be reshaped as a generative task with diffusion, and discard negative sampling.**
We sincerely hope that the initial exploration of Diff4Rec could enable recommendation task to embrace the benefits offered by the rapid development of diffusion models.
3. >*A negative size of 1 is too small for the model to perform well. In my experience, on the YooChoose dataset, a negative sample size of 49-99 would achieve much better result. Therefore the results reported in the paper might not be a fair comparison against baseline methods.*
We recognize the significance of a fair experimental setting, particularly the number of negative samples. In the literature of recommendation, **a fair experimental setting regarding the number of negatives could be that the number of negatives keeps the same across all models, addressing any potential bias arising from varying numbers of negatives.** Given that Diff4Rec discards negative samples, and the number of negatives of Diff4Rec can be seen as 0, we believe it is justifiable for other baselines to set their number of negatives as 1.
Meanwhile, we acknowledge the concern that more negative samples in classification-based baselines can improve performance, and we conduct experiments to search the number of negatives in [50, 60, 70, 80, 90, 100] following your advice. The results are as follows:
||YooChoose |-|KuaiRec|-|Zhihu|-
:--:|:--:|:--:|:--:|:--:|:--:|:--:
||HR@20(%)|NDCG@20(%)|HR@20(%)|NDCG@20(%)|HR@20(%)|NDCG@20(%)
SASRec|4.18|1.79|4.12|1.96|1.87|0.69
CL4SRec|4.64|1.93|4.31|2.11|2.10|0.75
Diff4Rec|4.78|2.23|5.26|4.11|2.26|0.79
Note that CL4SRec constructs many negative samples with augmentation, therefore more negatives result in limited improvement.
Best,
Authors