Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion

Sequential recommendation aims to recommend the next item that matches a user's interest, based on the sequence of items he/she interacted with before. Scrutinizing previous studies, we can summarize a common learning-to-classify paradigm -- given a positive item, a recommender model performs negative sampling to add negative items and learns to classify whether the user prefers them or not, based on his/her historical interaction sequence. Although effective, we reveal two inherent limitations:(1) it may differ from human behavior in that a user could imagine an oracle item in mind and select potential items matching the oracle; and (2) the classification is limited in the candidate pool with noisy or easy supervision from negative samples, which dilutes the preference signals towards the oracle item. Yet, generating the oracle item from the historical interaction sequence is mostly unexplored. To bridge the gap, we reshape sequential recommendation as a learning-to-generate paradigm, which is achieved via a guided diffusion model, termed DreamRec.Specifically, for a sequence of historical items, it applies a Transformer encoder to create guidance representations. Noising target items explores the underlying distribution of item space; then, with the guidance of historical interactions, the denoising process generates an oracle item to recover the positive item, so as to cast off negative sampling and depict the true preference of the user directly. We evaluate the effectiveness of DreamRec through extensive experiments and comparisons with existing methods. Codes and data are open-sourced at https://github.com/YangZhengyi98/DreamRec.

Paper

Similar papers

Peer review

Reviewer b6yV6/10 · confidence 3/52023-07-05

Summary

This paper presents an innovative paradigm shift in sequential recommendation systems. It departs from the traditional learning-to-classify approach, which includes negative sampling, to a learning-to-generate model, proposing a novel system called Diff4Rec, grounded on guided diffusion. This approach is predicated on the observation that users imagine an ideal or "oracle" item after several interactions with a recommendation system. The authors posit that Diff4Rec can generate these oracle items, modeling the underlying item-generation distribution through a diffusion process, and is not limited to a predefined candidate set. This approach allows for discarding negative samples, which previous models could not accomplish, as it models the data-generation process directly. The paper suggests that Diff4Rec effectively mirrors human behavior better than previous systems by capturing a user's preference more accurately and directly, avoiding the noisy or easy supervisions from negative samples. The authors evaluate Diff4Rec through various experiments, where it demonstrates consistent improvements over existing sequential recommendation models. In conclusion, this paper introduces a transformative methodology in sequential recommendation, potentially broadening the capabilities of recommendation systems.

Strengths

1. Raise a new learning-to-generate paradigm for recommendation using relative new diffusion model approach. 2. Good reproducibility: Code is released for readers; Hyperparameter settings are available. 3. Good to explain what each step is doing in the Algorithms. 4. Improvements over the baseline appear significant. But statistical significance analysis is missing.

Weaknesses

1. In terms of recommendation, an important and indispensable step is not discussed and stated in the algorithm. After an oracle item is generated, to fetch one / or some items to recommend to users, we need to find the top nearest items to the oracle item in the embedding space. Though this step is described in the Experiment, it is fully ignored in the Method section. I understand the Method part described mainly the learning-to-generate paradigm. But this step is needed to complete the recommendation task. Moreover, to retrieve top nearest items, how to encode properly the embedding space for items becomes also a relevant question. 2. Might be good to list out the main contributions of the work. 3. Please avoid abusing the footnotes. Footnotes are good for external links and websites. If you do think footnotes are providing important information, please try to integrate them in the main text instead, e.g. 3 and 4. And try to avoid the controversial remarks irrelevant to the main contents, like 2: people working on learning to rank may not fully agree on 2. 4. In table 1, if authors meant to have %, better to add (%) after HR@20 and NDCG@20. It’s easy to be confused, even when I read the caption.

Questions

1. A key success of the diffusion models in image generation is that it can encode some of undetermined features in the latent space so that random initializer helps to randomly sample those features in additional to the guidance signals. This implies, to a given user and interaction history, we can always generate different oracle items from different random seeds. How will this help on top item retrieval? For example, we can generate different oracle items and make a pooling to get an ensemble of oracles and then retrieve from that. How will this be different from retrieving one for each generated oracle? It would be very interesting for users to investigate on this end. 2. Can authors add one / more of the other diffusion model based recommendation model (19,20,21,36) results as a baseline? Tells us better the effect of with/without negative sampling.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

1. Authors may disagree on whether selecting the nearest items on the embedding space is a necessary step for recommendation. I could also imagine a recommendation with a generative item, for example, a LLM decoder to generate recommended results from the oracle item embedding generated with the diffusion result. Maybe authors could discuss how to fetch / generate the recommended item(s) from the oracle item in the main text.

Reviewer VUDN6/10 · confidence 3/52023-07-06

Summary

This paper describes a new learning-to-generate paradigm for sequential recommendation based on diffusion models. The authors discuss the limitations of previous approaches in sequential recommendation, where a recommender model learns to classify user preferences based on positive and negative item samples. The paper highlights two inherent limitations of this approach: (1) it differs from human behaviour, which involves knowing an ideal item and selecting potential matches, and (2) the classification is limited to the candidate pool, which may contain noisy or easily distinguishable negative samples, diluting the preference signals. This paper proposes Diff4Rec, a new learning-to-generate paradigm based on guided diffusion, which is able to circumvent the limitations of existing approaches. Empirical results on benchmark datasets show the effectiveness of Diff4Rec.

Strengths

- The authors analyzed the potential limitation of using negative sampling when training classification based recommendation models, and proposed a generation based framework using diffusion models, which is technically sound. - Diff4Rec is able to outperform a number of state-of-the-art baseline alternatives. - Code is available, which helps reproduce the results.

Weaknesses

- The authors analyzed the potential limitations of using negative sampling when training classification based recommendation models. In practice, however, a lot of models can be trained without using negative sampling at all. They directly use a softmax over all possible items to obtain the probability of each item being the next item. These softmax based approaches, in my opinion, are closely related to the spirit of Diff4Rec, i.e., by not relying on negative sampling and treat all items other than the next item as a whole. Thus the authors need to explain why Diff4Rec is superior to these approaches. - Diff4Rec can be very slow in both training and inference stages. In addition, the authors should analyze the time / computational complexity of Diff4Rec and present them in the paper. - The number of compared approaches is relatively small.

Questions

- In Diff4Rec, the max number of interactions is set to 10. Why not choosing a larger number of historical interactions? Is it due to Diff4Rec running too slow, or is it because of Diff4Rec not performing well when the sequential length is large? - The abstract in the OpenReview console is not consistent with the submitted paper. In OpenReview, the model is termed SRDiff, while in the paper it is termed Diff4Rec.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have done a good job discussing the limitations of Diff4Rec. Diff4Rec has a very high computational cost, which limits its applicability. When considering using Diff4Rec in application, it is important to analyze the tradeoff between computational need and recommendation performance. However, Diff4Rec offers a new view of reshaping sequential recommendation as an item generation task, which has the potential to inspire more research in this direction. This outweighs the existing limitations of Diff4Rec.

Authorsrebuttal2023-08-18

We are looking forward to your further comments.

Dear Reviewer VUDN, Thank you again for your insightful feedback on our submission, particularly your suggestions to 1) **explain why Diff4Rec is superior to softmax-based approaches**, 2) **discuss the time / computational complexity of Diff4Rec in the paper**, 3) **verify our method with a larger number of historical interactions than 10**, and 4) **incorporate more baseline approaches**. These valuable suggestions better strengthen the quality of our work. The deadline of the discussion stage is approaching, and we are looking forward to your further comments. We sincerely hope that these improvements will be taken into consideration. If we have properly addressed your concerns, we would deeply appreciate it if you could kindly re-evaluate our paper. If you have further concerns, please let us know and we remain open and would be more than happy to actively discuss them with you. Best, Authors

Reviewer VUDN2023-08-18

I would like to thank the authors for the detailed explanation. After reading other reviewers' comments and the authors' rebuttal, I will keep my original score. Firstly, softmax-based approaches should also be considered as generative methods since they directly models the probability of a target item being the next item. In that way, the distinction between softmax and diffusion is really marginal. The authors also argued that softmax-based approaches explicitly used negative samples, while diff4rec does not. In fact, both softmax and diffusion implicitly use negative sampling. If a diffusion model does not implicitly use negative sampling, then the model will simply collapse. Another potential problem of the evaluation protocol is that, the authors said "For all baselines, we conduct negative sampling from the uniform distribution at the ratio of 1: 1, which is not conducted in Diff4Rec." This is not really a good practice for training sequential recommendation models. A negative size of 1 is simply too small for the model to perform well. In my experience, for example, on the YooChoose dataset, a negative sample size of 49-99 would achieve much better result. Therefore the results reported in the paper might not be a fair comparison against baseline methods.

Authorsrebuttal2023-08-18

Responce to your further concerns.

Dear Reviewer VUDN, Thank you very much for your valuable feedback. We sincerely hope that our following response could properly address your concerns. 1. >*Softmax-based approaches should also be considered as generative methods since they directly model the probability of a target item being the next item. In that way, the distinction between softmax and diffusion is really marginal.* We do agree that softmax directly models the probability of a target item being the next item. Meanwhile, we would like to kindly emphasize that **softmax and generative models are different due to their adherence to distinct paradigms (discriminative and generative respectively)**. Specifically, generative models (GANs, VAEs and diffusion models, e.t.c) directly model the underlying data generation distribution by learning the map from Gaussian distribution and the underlying distribution. **In contrast, softmax is commonly employed in discriminative models, since it models the probability distribution over a set of discrete classes, with limited presence in the literature of generative models**. Besides, softmax is limited to the discrete candidate set, whereas diffusion can generate samples beyond the candidate. Therefore, we would respectfully emphasize that softmax and diffusion models are quite distinctive. 2. >*The authors argued that softmax-based approaches explicitly use negative samples, while diff4rec does not. In fact, both softmax and diffusion implicitly use negative sampling. If a diffusion model does not implicitly use negative sampling, the model will simply collapse.* We agree that softmax implicitly uses negative sampling (we feel sorry about the potential confusion caused by that it should be 'implicitly' instead of 'explicitly' in how softmax uses unobserved interactions in the table of Point 1 in the rebuttal). Meanwhile, we would kindly emphasize that **negative sampling is not used in Diff4Rec, either explicitly or implicitly**. The primary objective of Diff4Rec is: $L_{t-1} = \mathbb{E}_{e_n\^0, \epsilon}\left[\frac{\bar\alpha\_{t-1}}{2\tilde{\beta}\_t} ||e\_n^0 - f\_\theta(\sqrt{\bar\alpha\_t} e_n^{0} + \sqrt{1-\bar\alpha\_t}\epsilon, c\_{n-1}, t)||^2\right] + C,$ where $e_n^0$ is sampled from **only observed interactions**. We have also provided the code and training algorithm, and we hope these materials could help address your concern. **Even if negative sampling is not used (implicitly or explicitly) in Diff4Rec, it would not collapse, which is one of the contributions of this work**. The reason can be attributed to the fundamental paradigm shift: the paradigm of Diff4Rec is learning-to-generate through diffusion, instead of traditional learning-to-classify. Specifically, Diff4Rec distinctly models the underlying generation distribution of observed interactions with the power of diffusion, which does not require negative samples. However, previous sequential recommenders adhere to the learning-to-classify paradigm and distinguish positive and negative samples, which strictly requires negative sampling. **We do understand that recommendation has long been recognized as a discriminative task requiring negative sampling. Yet our work does show that sequential recommendation can be reshaped as a generative task with diffusion, and discard negative sampling.** We sincerely hope that the initial exploration of Diff4Rec could enable recommendation task to embrace the benefits offered by the rapid development of diffusion models. 3. >*A negative size of 1 is too small for the model to perform well. In my experience, on the YooChoose dataset, a negative sample size of 49-99 would achieve much better result. Therefore the results reported in the paper might not be a fair comparison against baseline methods.* We recognize the significance of a fair experimental setting, particularly the number of negative samples. In the literature of recommendation, **a fair experimental setting regarding the number of negatives could be that the number of negatives keeps the same across all models, addressing any potential bias arising from varying numbers of negatives.** Given that Diff4Rec discards negative samples, and the number of negatives of Diff4Rec can be seen as 0, we believe it is justifiable for other baselines to set their number of negatives as 1. Meanwhile, we acknowledge the concern that more negative samples in classification-based baselines can improve performance, and we conduct experiments to search the number of negatives in [50, 60, 70, 80, 90, 100] following your advice. The results are as follows: ||YooChoose |-|KuaiRec|-|Zhihu|- :--:|:--:|:--:|:--:|:--:|:--:|:--: ||HR@20(%)|NDCG@20(%)|HR@20(%)|NDCG@20(%)|HR@20(%)|NDCG@20(%) SASRec|4.18|1.79|4.12|1.96|1.87|0.69 CL4SRec|4.64|1.93|4.31|2.11|2.10|0.75 Diff4Rec|4.78|2.23|5.26|4.11|2.26|0.79 Note that CL4SRec constructs many negative samples with augmentation, therefore more negatives result in limited improvement. Best, Authors

Reviewer RbjC5/10 · confidence 3/52023-07-10

Summary

The paper presents a new approach Diff4Rec, a guided diffusion model for sequential recommendations. The paper could be divided in three main parts: 1. Problem formulation and method: It describes the sequential recommendation as oracle item generation and then explains how diffusion is applied. 2. Experiments: Describes the datasets used and various experiments done. Moreover, they have provided code also. 3. Results: Presents the results from their experiment. The results are much superior compared to the previous methods.

Strengths

The paper presents a new idea in the learning-to-generate paradigm, which aimed to used diffusion process. They have supported the results with exhaustive set of experiments and validated the results on multiple datasets.

Weaknesses

1. The paper doesn't show the impact of negatives on the recommendations 2. Would it be possible to present the A/B test results. I suspect the recommendations would over hinge on the users history while recommending for a longer period of time..

Questions

1. Noob q: As per my understanding, during the inference time we will get embedding for nth item as a oracle item. How do we recover what item to recommend based on the predicted embedding? 2. Do we have any results from an A/b test?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

A/B testing on real system would have helped to validate the proposed idea.

Authorsrebuttal2023-08-18

We are looking forward to your further comments.

Dear Reviewer RbjC, Thank you again for your valuable feedback on our submission, particularly your suggestions to 1) **highlight the inference of recommendation results**, and 2) **demonstrate that the recommendations would not over-hinge on the users' history with A/B test**. We also try our best to address your concern about **the impact of negatives on the recommendations from the perspectives of negative societal impacts and the impact of negative sampling**. These insightful suggestions better strengthen the quality of our paper. The deadline of the discussion stage is approaching, and we are looking forward to your further feedback. We sincerely hope that these improvements will be taken into consideration. If we have properly addressed your concerns, we would be grateful if you could kindly re-evaluate our paper. If you have additional concerns, please let us know and we remain open and would be more than happy to actively discuss them with you. Best, Authors

Authorsrebuttal2023-08-15

Supplement to Point 6(the additional baseline [21])

As a supplement to Point 6, we have implemented model [21] based on the paper's details, and the result is shown as follows: ||YooChoose | YooChoose | KuaiRec | KuaiRec | Zhihu | Zhihu :--: |:--: | :--: | :--: | :--: | :--: | :--: ||HR@20(%) | NDCG@20(%) | HR@20(%) | NDCG@20(%) | HR@20(%) | NDCG@20(%) DSR [21] | 4.12$\pm$0.06|1.81$\pm$0.05|3.79$\pm$0.03|1.64$\pm$0.03|1.80$\pm$0.04|0.65$\pm$0.02 Note that the results of models [19], [20], and [36] are provided in Table 4 of the PDF. We are anticipating a deeper discussion with you!

Reviewer b6yV2023-08-15

Thank you, authors, for the detailed response to my questions and suggestions. Hope some of the discussions would be useful to the paper and will be integrated to it in the final version. Given the potential impact of the new recommendation framework proposed by the work, I'll continue voting for accepting the paper.

Authorsrebuttal2023-08-16

Dear Reviewer b6yV, Thank you for your time and effort in reviewing our paper and providing valuable feedback. We appreciate your insights, and we will try our best to integrate the discussion into our revision. We sincerely hope that our response has properly addressed your concerns. If so, we would deeply appreciate it if you could re-evaluate our work and raise your score slightly. If you have further concerns, we will continue actively responding to your comments and/or questions. Thank you again for your attention. Best, Authors

Authorsrebuttal2023-08-21

Thank you for your time and effort.

Dear AC and Reviewers, We would like to express our sincere gratitude for your dedicated time and efforts in evaluating our submission. In essence, **Diff4Rec reshapes sequential recommendation as a learning-to-generate task with diffusion models, in contrast to the conventional perception of recommendation as a learning-to-classify (or discriminative) task**. Therefore, Diff4Rec has the potential to explore the underlying distribution of item space, enabling the generation of items beyond the confines of predefined candidates. This sets Diff4Rec apart from classification-based recommenders, which are constrained by discrete candidate set. Having received so many valuable suggestions in the rebuttal and discussion phases, we are committed to incorporating the reviewers' suggestions into our manuscript. These insightful suggestions would definitely strengthen the quality of our paper! We are particularly appreciative of the reviewers' recognition that Diff4Rec brings the potential impact of the new recommendation framework , and the potential to inspire more research in this direction. Once again, we sincerely express our gratitude to you, our AC and reviewers, for guiding us through this insightful review process! Appreciatively, Authors

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC