Diffusion Models Meet Contextual Bandits

We thank the reviewer for their thoughtful feedback and recognition of the novelty, clarity, and rigor of our work. Below, we address the key concerns. --- ### Concern 1: When the true prior is not a diffusion model **Reviewer’s Comment:** The reviewer questions whether the algorithm performs effectively when the true prior is not a diffusion model. **Response:** dTS performs strongly even when the true prior is not a diffusion model. As shown in our experiments (Section 5, Figure 4), dTS consistently outperforms LinTS although the pre-trained diffusion model is only an apprixmation of the true prior distribution which is not a diffusion model. Specifically, Figures 4(a, b) show that the regret ratio of LinTS to dTS exceeds 1 across all tested diffusion model depths ($L \in [2, 100]$) and pre-training sample sizes ($[10, 50000]$), and in all these scenarios the true prior distribution is not a diffusion model. Similarly, dTS outperforms LinTS in the MovieLens experiments, where the true prior is also not a diffusion model. --- ### Concern 2: Theoretical analysis limited to the linear case **Reviewer’s Comment:** The theoretical statements are valid only for linear link functions, and it is unclear whether performance gaps between linear and non-linear link functions exist, potentially impacting applicability. **Response:** The linear case provides a tractable framework for establishing theoretical guarantees. Extending the analysis to non-linear link functions is challenging and remains future work. However, our empirical results (Section 5) demonstrate that dTS performs strongly in non-linear settings, consistently outperforming baselines, which supports its broader applicability. --- ### Concern 3: The combined focus on contextual bandits and dependent arms **Reviewer’s Comment:** The reviewer finds it challenging to isolate the impact of the diffusion model due to the combined focus on contextual bandits and bandits with dependent arms. **Response:** Theoretically, we show improvements in settings with dependent arms, as our analysis assumes linear link functions and leverages these dependencies. In practice, however, diffusion models are beneficial even when arms are independent, as they capture the complex distribution of action parameters. --- ### Concern 4: The necessity of using a diffusion model **Reviewer’s Comment:** The reviewer questions whether a diffusion model is necessary, particularly when it is already pre-trained, for handling dependent arms in contextual bandits. **Response:** The diffusion model is essential for capturing complex action parameter distributions that standard priors (e.g., Gaussians) cannot handle. While the pre-trained diffusion model serves as a strong starting point, combining it with Thompson Sampling (TS) enables adaptive exploration of high-reward regions. Efficient posterior updates and sampling fine-tunes the diffusion model, leading to better decision-making. This integration is critical; without it, performance deteriorates, as evidenced by dTS’s initial increase in cumulative regret (when relying solely on the pre-trained diffusion model). Over time, cumulative regret stabilizes and converges as TS updates the model. --- We’re grateful for the detailed feedback, which allowed us to strengthen our work through additional clarifications and experiments. If we have adequately addressed the reviewer’s concerns, we kindly request a re-evaluation of our work and a potential increase in score/confidence. For any unresolved issues, we’re happy to provide further clarifications. Thank you very much!

We thank the reviewer for their detailed feedback and for highlighting the novelty and practical insights of our work. Below, we address the key concerns. --- ### Concern 1: Interpreting the setting as a generalized linear contextual bandit **Reviewer’s Comment:** The reviewer suggests that concatenating action parameters reduces the problem to a generalized linear contextual bandit of dimension $Kd$, with regret $\tilde{O}(\sqrt{Kdn})$. **Response:** This observation overlooks the critical aspect of computational efficiency. As discussed in Section 4.1, treating the problem this way results in computational complexity $\mathcal{O}(K^3d^3)$ and spatial complexity $\mathcal{O}(K^2d^2)$, which are prohibitively expensive for large $K$. In contrast, dTS achieves $\mathcal{O}((K+L)d^3)$ and $\mathcal{O}((K+L)d^2)$, where $L$ is the diffusion model depth. Since $K \gg L$ in practice, dTS offers significantly better scalability and efficiency. Beyond computational advantages, dTS leverages the rich prior distribution captured by (non-linear) diffusion models, resulting in improved performance over standard generalized linear contextual bandit algorithms, as demonstrated in our experiments. --- ### Concern 2: Theoretical analysis assuming linear link functions **Reviewer’s Comment:** The reviewer posits that linear link functions reduce the problem to a trivial case, as combinations of linear functions remain linear. **Response:** We agree that linear link functions simplify the analysis. However, even in the linear case, dTS outperforms standard LinTS (see Section 4.1). For linear link functions, similar results to “dTS with any $L$” can be achieved using “dTS with $L=1$” by marginalizing all latent parameters $\psi_\ell$ for $\ell \neq 1$ (except $\psi_1$). This marginalization, however, is only feasible in the linear case and becomes intractable for non-linear link functions. We stress however that Section 4 only complements the paper by analyzing a simplified instance of dTS to provide theoretical insights. The broader contributions of the paper extend beyond this setting. While extending the theoretical analysis to non-linear link functions is challenging and remains future work, dTS is designed to handle any (non-linear) link function. Our empirical results confirm its superior performance over baselines in both linear and non-linear link functions. --- ### Concern 3: Clarity on dTS inputs **Reviewer’s Comment:** The reviewer found certain parts unclear, such as whether the covariance matrices $\Sigma_\ell$ or the link function $f_\ell$ are revealed to the agent. **Response:** Algorithm 1 specifies that both the covariance $\Sigma_\ell$ and link function $f_\ell$ are provided as inputs and known to the agent. This is also clarified in the text (starting from line 090). This assumption is mild, as these parameters are typically accessible in pre-trained diffusion models. --- We’re grateful for the detailed feedback, which allowed us to strengthen our work through additional clarifications and experiments. If we have adequately addressed the reviewer’s concerns, we kindly request a re-evaluation of our work and a potential increase in score/confidence. For any unresolved issues, we’re happy to provide further clarifications. Thank you very much!

We thank the reviewer for their detailed feedback and recognition of the novelty and practical effectiveness of our proposed algorithm. Below, we address the key concerns and suggestions. --- ### Concern 1: Presentation and justification of assumptions **Reviewer’s Comment:** The presentation of assumptions is unclear, and certain assumptions (e.g., (A0) and (A2)) require better justification. The numbering should start from 1 instead of 0, and Assumption (A0) should be separated into two distinct parts. **Response:** We appreciate this suggestion and have revised the manuscript accordingly. Assumptions now start from (A1), and Assumption (A1) (previously (A0)) has been separated into two parts for clarity. Additionally, Assumption (A2) is now explicitly justified: when a pre-trained diffusion model (with linear link functions) is used as a prior, (A2) is automatically satisfied. This has been clarified in the revised manuscript. --- ### Concern 2: Comparison of Bayes regret bounds with prior work **Reviewer’s Comment:** The Bayes regret bound in Theorem 4.1 compared to existing bounds. **Response:** The regret bound in Theorem 4.1 is tighter than previous bounds. Specifically, Bayes regret bounds scale as $\mathcal{O}(\sqrt{dKn})$ for finite action spaces and $\mathcal{O}(d\sqrt{n})$ for infinite action spaces. This explains the $\mathcal{O}(\sqrt{d})$ improvement over Russo and Van Roy (2014), who assumed an infinite action space. But, Theorem 4.1 has smaller constants and provides a tighter bound compared to Russo and Van Roy (2014). It is also tighter than the frequentist bound for linear Thompson sampling (e.g., [1]). This is due to the fact that the constants are smaller in our case compared to prior studies. Moreover, in the non-contextual setting ($d=1$), our bound scales as $\mathcal{O}(\sqrt{K})$, matching the rate by Agrawal and Goyal (2023b) as reported in the review (although we could not identify the exact paper referenced). In general, Bayes regret bounds are inherently tighter than frequentist bounds, which aligns with the expectation that Theorem 4.1 outperforms worst-case bounds. --- ### Concern 3: Impact of $L$ **Reviewer’s Comment:** The section on “Why the bound increases with $L$?” (Line 268) is unclear, and the explanation seems contradictory to the theoretical results. **Response:** The theoretical results predict that regret increases with $L$ when the true prior distribution matches a diffusion model of depth $L$, as increasing $L$ reflects a more complex action parameter distribution and hence a more complex bandit problem. This behavior is captured by our regret bound. In practice, when $L$ is small, the pre-trained diffusion model may be too simple to capture the true prior distribution, violating the assumptions of our theory. Increasing $L$ improves the pre-trained model’s quality, reducing regret. Once $L$ is large enough and the pre-trained model adequately captures the true prior distribution, the theoretical assumptions hold, and regret begins to increase with $L$, as predicted. This is validated empirically in Figure 4(b). --- ### Concern 4: Table of notations **Reviewer’s Comment:** A table of notations is necessary to improve clarity. **Response:** We have added a table of notations at the beginning of the appendix to enhance readability and assist readers in following the mathematical presentation. --- ### Concern 5: Novelty of sparsity assumption **Reviewer’s Comment:** It is debatable whether adding a sparsity assumption qualifies as novel unless it introduces new insights or milder conditions. **Response:** We agree that sparsity assumptions are not inherently novel unless they provide new insights. Our insight is that regret scales only with $d_\ell$ (the non-sparse dimensions of the mixing matrices) rather than the full dimension $d$. We have rephrased the relevant statement in the revised manuscript to emphasize this contribution. --- We’re grateful for the detailed feedback, which allowed us to strengthen our work through additional clarifications and experiments. If we have adequately addressed the reviewer’s concerns, we kindly request a re-evaluation of our work and a potential increase in score. For any unresolved issues, we’re happy to provide further clarifications. Thank you very much! [1] Abeille, Marc, and Alessandro Lazaric. "Linear thompson sampling revisited." Artificial Intelligence and Statistics. PMLR, 2017.

We thank the reviewer for the follow-up questions and for seeking further clarification. Below, we address each question in detail. --- ### Question 1: Regret bound comparison in the case $d = 1$ **Reviewer’s Question:** When $d = 1$, isn't your Bayes regret bound still $\mathcal{O}(\sqrt{(K+L)n})$? This appears worse than the standard $\mathcal{O}(\sqrt{Kn})$ bound for non-contextual multi-armed bandits (MABs), such as those presented in Agrawal and Goyal (2013b). **Response:** For $d=1$, our Bayes regret bound is $\mathcal{O}((K+L)n)$, but constants are ignored in this form. In Bayes regret, constants are crucial as they showcase the benefits of using informative priors (e.g., small $\sigma_\ell^2$ in our case). It is hard to compare tightness theoretically, but we can plot the bounds in their explicit expressions. Agrawal and Goyal (2013b) do not provide explicit constants in their results. Therefore, we compared our bound to existing frequentist bounds for Thompson sampling in linear bandits (e.g., [1]) in Appendix E.4 (Figure 7(a)) and found that our bound is much tighter. Similar behavior is expected for the MAB setting. Furthermore, our bound is tighter than standard Bayes regret bounds for LinTS, as illustrated in Appendix E.4 (Figure 7(b)). These comparisons highlight the advantages of incorporating structured priors into dTS. **Important Note:** While these comparisons are insightful, regret bounds derived for linear bandits are generally less tight when applied to MAB settings compared to bounds specifically derived for MABs. This applies to Question 2 as well. --- ### Question 2: Fixed contexts over time and motivation for dTS **Reviewer’s Question:** When $d > 1$ but the context vectors are fixed over time, can't we treat each arm independently and apply standard MAB algorithms to achieve $\mathcal{O}(\sqrt{Kn})$ regret? In contrast, your approach seems to incur $\mathcal{O}(\sqrt{(K+L)dn})$ regret. Is there still motivation to use dTS in this case? **Response:** We agree that in this specific setting, the $d$ term remains in our bound. This occurs because our regret bounds are derived for linear bandits, and translating them to fixed-context scenarios does not eliminate the context dimension $d$. But this is true for regret bounds in the literature that were derived in linear bandits; translating these to the specific fixed-context case doesn’t make the context dimension $d$ disappear However, by adapting our proof techniques for the MAB setting, we can derive bounds independent of $d$. In such cases, posterior variances replace posterior covariances in the analysis, effectively removing the dependence on $d$. We did not do this since we focused on generalized linear bandits. For both questions Q1 and Q2, in practice, using dTS is more beneficial in high-dimensional problems with complex dependencies. Thus, in simple MAB problems, one might use simpler models. --- ### Question 3: Sweet spot for $L$ and practical recommendations **Reviewer’s Question:** Regarding your comment that "increasing $L$ improves the pre-trained model's quality, reducing regret," and the empirical results in Figure 4(b): Is there a sweet spot for $L$, and how should practitioners choose $L$? **Response:** Yes, as shown in Figure 4(b), performance improves with $L$ until approximately $L=40$, after which it declines. The optimal value of $L$ depends on the application. Practitioners can empirically tune $L$ by selecting the smallest value that accurately captures the offline data distribution. This tuning process can be performed offline before integrating the diffusion model prior into dTS, ensuring both computational efficiency and improved performance. --- We hope this response addresses your questions and clarifies our results. Please let us know if further clarifications are needed. Thank you for your thoughtful feedback and prompt response! We greatly appreciate it. [1] Abeille, Marc, and Alessandro Lazaric. "Linear thompson sampling revisited." Artificial Intelligence and Statistics. PMLR, 2017.

Thank you for your response. However, I feel that my primary concern remains unresolved. As I stated in my earlier comment: *“Even if we set aside the dependence on $d$, along with the question in the previous paragraph, I’m more **concerned about the additional $\sqrt{Ln}$ term in your bound**”* My main question was and is about the extra $\sqrt{Ln}$ dependence in your proposed method’s regret bound. From your response, aside from the acknowledgment that *“in simple MAB problems, one might use simpler models,”* it is unclear how the additional dependence on $L$ aligns with the practical utility of dTS in such settings. The cases with $d = 1$ and $d > 1$ with fixed contexts—both of which are special cases of your problem—highlight that your method incurs sub-optimal regret due to the additional $L$-dependent term. Simply stating that *“using dTS is more beneficial in high-dimensional problems with complex dependencies”* feels vague and lacks specificity. It is critical to clearly define the regimes where your method is advantageous and where it is not. I acknowledge that as $L$ increases in the prior, the problem inherently becomes more complex, and the regret bound naturally increases. However, your argument in the section *“Why the bound increases with $L$?”*—and even in your response above—that there is an optimal $L$ in practice does not align with your theoretical result, where the regret monotonically increases with $L$. This statement is confusing and risks misrepresenting the theoretical findings. I strongly suggest removing or rephrasing the statement, *“Increasing $L$ improves the pre-trained model’s quality, reducing regret,”* as your definition of Bayes regret is defined with respect to a known prior (hence "Increasing $L$ ... reducing regret" does not make sense). If you wish to discuss the practical behavior of $L$ (when $L$ is not known?), such discussion belongs in the experimental section and should not be conflated with the theoretical analysis. Finally, regarding practical applications, let’s consider a scenario where you are solving a contextual bandit problem with $K$ parameters from scratch, with no pre-existing data (a typical bandit setting). How would you recommend choosing $L$ in such a case? In many practical scenarios, your exact prior or $L$ may not be known in advance. I look forward to further clarifications.

We would like to thank you for engaging in this discussion; we greatly appreciate your thoughtful feedback and the opportunity to clarify our approach. ### **(I) Dependence on $L$ Does Not Make Bayes Regret Suboptimal** --- #### **1) When Comparing Bayes Regret Bounds with Each Other** Let us start by clarifying that Bayes regret should be understood as follows: when the true prior is $P_0$, the Bayes regret is $\mathcal{O}(f(n, K, d, P_0))$, where the bound $f(n, K, d, P_0)$ depends on the horizon $n$, problem parameters $K, d$, and the true prior $P_0$ (more precisely, it depends on the variance of the true prior $P_0$). Therefore, when the true prior $P_0$ has high variance, the problem becomes more challenging, and the bound increases naturally. The $L$ term is directly related to the variance of the diffusion model prior, and this is why it appears. However, this is not problematic, as we explain next. For dTS, the true prior $P_0$ is a diffusion model, and the regret bound is: $$ \mathcal{O}\left(\sqrt{nd \left(K \sigma_1^2 + \sum_{\ell=1}^L \sigma_{\ell+1}^2\right)}\right), $$ where we have dropped dependence on $\sigma_{max}$ to simplify the explanation. Indeed, we observe an additional $L$ term, but this arises because the overall variance of this prior is roughly $\sum_{\ell=1}^{L+1} \sigma_{\ell+1}^2$. For LinTS with a Gaussian prior that marginalizes over all latent parameters, the prior variance is roughly $\sum_{\ell=1}^{L+1} \sigma_\ell^2$, and the Bayes regret becomes: $$ \mathcal{O}\left(\sqrt{ndK \sum_{\ell=1}^{L+1} \sigma_\ell^2}\right). $$ Hence, the $L$ term also appears in the LinTS regret bound. In fact, dTS has a better dependency on $L$ because it only multiplies $K$ by $\sigma_1^2$, whereas in LinTS, $K$ is multiplied by the much higher constant $\sum_{\ell=1}^{L+1} \sigma_\ell^2$. The $L$ dependency is thus a consequence of the prior variance in a diffusion model and will also appear for LinTS Bayes bounds. Note that comparing Bayes regret bounds with different prior variances is not meaningful; for example, comparing LinTS with a small prior variance $v$ to dTS with a much larger prior variance is not meaningful. This is because we could reduce the variances in the diffusion model prior so that their sum $\sum_{\ell=0}^L \sigma_\ell^2$ is smaller than the variance used in LinTS. Actually, the dependence on $L$ can disappear with proper choices of $\sigma_\ell$. This is why to compare Bayes regret bounds, we compare methods whose true priors have the same variance (e.g., we compare methods in the same problem difficulty). A rigorous comparison of our bound with that of LinTS is given in Section 4.1. --- #### **2) When Comparing Bayes Regret Bounds with Frequentist Bounds** First, we do not believe it is rigorous to compare Bayes regret bounds with frequentist ones, as they operate in very different settings, and it is common practice not to compare them. However, if we make such a comparison and include $L$ in it, then all other constants (including variances) must also be included for a rigorous analysis. When this is done, our bound is much tighter (Appendix E.4). Of course, if we only care about asymptotic behavior and include $L$, the presence of $L$ may make it appear as if the bound is loose. However, as shown in our results (Appendix E.4), our bound is significantly smaller than the frequentist bound for LinTS and the Bayes regret bound for LinTS. --- ### **(II) Practical Use of Diffusion Models** If no offline data or pre-trained diffusion model is available, simpler models may be more appropriate. We will clarify that dTS relies on a pre-trained diffusion model, either from an existing source or trained on offline data (as done in our MovieLens experiment). However, this is not a strong assumption. In practical applications, offline data is often available, enabling us to pre-train a diffusion model. There are also numerous pre-trained diffusion models available for domains similar to the bandit problem (e.g., pre-trained models on product images for recommendation problems). The choice of $L$ depends on the problem, and we select the smallest $L$ that achieves the best empirical results on the offline data. This process is independent of our work and follows standard guidelines for pre-training diffusion models. To address potential confusion, we will move the discussion of the practical behavior of $L$ to the experiments section. Many thanks for this suggestion. --- We have attempted to explain this from first principles regarding how Bayes regret bounds should be compared (and that we should account for constants in our comparisons). We hope this explanation is clear, and we are keen to elaborate further if needed. Thank you again for engaging in this discussion; we greatly appreciate it.

Thank you for your response. However, I believe there is a misunderstanding regarding my original question, as evident in the authors’ response **(I) Dependence on $L$ Does Not Make Bayes Regret Suboptimal**. My concern was never about the comparison between LinTS and your method (or frequentist vs. Bayes regrets). Instead, my focus was on the apparent sub-optimality of your proposed method in special cases, such as $d = 1$ or $d > 1$ with fixed context. To clarify, throughout the discussion, I emphasized that in special cases (e.g., $d = 1$ or $d > 1$ with fixed context), the problem appears to admit a $\sqrt{Kn}$ **Bayes regret** bound when solved as a multi-armed bandit (MAB), while your proposed solution incurs a $\sqrt{(K+L)n}$ regret bound for any $L \geq 1$. This creates a gap in such cases, which is the sub-optimality I referred to—not the gap between LinTS and your method (or not the difference between frequentist and Bayes regrets). To further elaborate, consider a case when $d = 3$ with $\mathcal{X} = $ {$(1,2,0), (0,1,2)$}. Suppose the nature selects $X_t = (1,2,0)$ for odd $t$ and $X_t = (0,1,2)$ for even $t$ (this selection of nature is unknown to the algorithm a priori, but $X_t$ is revealed at each time). Given the diffusion prior as defined in your method, this problem setting satisfies your problem set-up. For this, one can treat this as two distinct instances of MABs: when $X_t = (1,2,0)$ it is one instance, and when $X_t = (0,1,2)$ it is the other instance. Each instance can theoretically be solved with a $O(\sqrt{Kn})$ Bayes regret bound (with time horizon effectively halved for either instance, which is just a constant factor, also suppose I do not care about the dependence on $d$ here). This approach avoids any dependence on $L$ whereas your method will still have $L$ dependence. I would like to clarify whether this reasoning is incorrect. Simple “Yes” or “No” would be fine. --- **[Minor comments, not directly related to the primary concern]** Regarding the comparison between Bayes regret bounds and frequentist bounds, I want to clearly clarify my statement about Theorem 4.1. When I noted that the regret bound is “worse than both the non-contextual result in Bayes regret and the worst-case regret bound,” I meant that Theorem 4.1 is sub-optimal even in terms of Bayes regret. The reason I mentioned the frequentist regret bound was to highlight that Bayes regret is typically expected to be sharper than (or at most equal to) the frequentist regret bound for the same problem (same problem but the regret definition differ by the expectation over prior), given the weaker nature of Bayes regret. However, my primary concern remains the sub-optimality of Theorem 4.1 in terms of Bayes regret itself, particularly in the scenarios outlined above. You do not need to answer this part. This is just my clarification, just so that the authors understand the point. I hope this clarification resolves any misunderstanding and look forward to your thoughts on these points.

We thank the reviewer for their detailed feedback, recognition of the clarity and coherence of our work, and thoughtful suggestions for improvement. Below, we address the main concerns and suggestions. --- ### Concern 1: Experimentation in scenarios with non-diffusion model priors **Reviewer’s Comment:** The performance of dTS in settings where the true prior is not a diffusion model is only compared to LinTS, which cannot capture correlations among actions. A fairer comparison would include algorithms like "Vits: Variational Inference Thompson Sampling" or DiffTS from Hsieh et al. (2023). **Response:** We did not include DiffTS as it focuses on the multi-armed bandit setting and is not directly applicable to the contextual bandit framework. This limitation is acknowledged in their conclusion, where they propose extending DiffTS to contextual bandits as a challenging future direction. As such, neither empirical nor theoretical comparisons are feasible, as DiffTS does not work in our setting and lacks a regret bound. We are currently running MovieLens experiments against ViTS and will share results upon completion. --- ### Concern 2: Additional related work and references **Reviewer’s Comment:** The reviewer points out missing references. **Response:** We have cited Neu et al. (2022) and Gouverneur et al. (2023) and Dani et al. (2008) and included references to support the claim regarding the suitability of Bayes regret for evaluating informative priors. --- ### Concern 4: Minor revisions for clarity and consistency **Reviewer’s Comment:** The reviewer suggests minor edits for improved clarity. **Response:** We have incorporated all the suggested edits to improve clarity and consistency in the manuscript. --- We’re grateful for the detailed feedback, which allowed us to strengthen our work through additional clarifications and experiments. If we have adequately addressed the reviewer’s concerns, we kindly request a re-evaluation of our work and a potential increase in score. For any unresolved issues, we’re happy to provide further clarifications. Thank you very much!

We thank the reviewer for their thoughtful feedback and recognition of the strengths and contributions of our work. Below, we address the key concerns and questions raised. --- ### Concern 1: Reliance on pre-trained diffusion models **Reviewer’s Comment:** The method’s performance depends heavily on the accuracy of the pre-trained diffusion model. Misspecified priors could compromise posterior approximations and regret bounds. **Response:** We agree that dTS relies on the quality of the pre-trained diffusion model. However, dTS mitigates this limitation by combining the diffusion model prior with Thompson Sampling, enabling adaptive updates based on observed rewards. As shown in our experiments, dTS outperforms baselines even when the prior is misspecified, leveraging observed data to refine decision-making. --- ### Concern 2: Limited discussion on alternative approximation techniques **Reviewer’s Comment:** There is limited discussion of alternative techniques, such as variational inference or Monte Carlo methods, that could offer more flexible or accurate approximations for non-linear reward distributions. **Response:** We appreciate this suggestion and acknowledge the potential of alternative techniques [1,2]. However, these methods often involve higher computational costs, making them less practical for large-scale action spaces. dTS focuses on efficient posterior updates and sampling. For linear rewards, our method provides a closed-form solution, while for non-linear rewards, only MLE optimization is required. In contrast, variational inference introduces additional bias by approximating the posterior as Gaussian, whereas dTS preserves the richer structure of diffusion models. Monte Carlo methods, while flexible, incur significant computational overhead, particularly in bandit settings. We are also running ViTS [3] on the MovieLens dataset and will share results upon completion. --- ### Response to Questions: 1. **Impact of the link function on posterior approximation and computational efficiency** The complexity of evaluating the link function affects posterior approximations minimally. For linear link functions, approximations are exact. Quantifying the impact of non-linear link functions on approximation quality is difficult, but empirical results show strong performance, especially as non-linearity increases. 2. **Changes in regret bounds with non-linear transformations** While our theoretical analysis focuses on linear link functions, regret bounds for non-linear transformations are challenging to derive. Empirical results demonstrate robust performance in non-linear cases, with greater gaps over baselines as non-linearity increases. 3. **Effect of the number of layers $L$ on convergence** The theoretical results predict that regret increases with $L$ when the true prior matches a diffusion model of depth $L$, reflecting the increased complexity of the action parameter distribution. Empirically, regret initially decreases as $L$ increases due to improved pre-trained diffusion models. Once $L$ is sufficiently large and the pre-trained model captures the true prior, regret begins to increase with $L$, consistent with theoretical predictions (Figure 4b). 4. **Impact of sparsity assumptions** Sparsity assumptions do not affect computational efficiency or posterior derivations. However, they reduce regret, as dTS’s regret depends only on the non-zero submatrix dimensions rather than the full matrix dimensions. --- We thank the reviewer for their detailed feedback, which has strengthened our work through additional clarifications and experiments. If our responses address your concerns, we kindly request a re-evaluation and a potential increase in score. We are happy to provide further clarifications if needed. Thank you! --- **References:** [1] Xu, Pan, et al. "Langevin Monte Carlo for contextual bandits." *International Conference on Machine Learning.* PMLR, 2022. [2] Karbasi, Amin, et al. "Langevin Thompson Sampling with logarithmic communication: Bandits and reinforcement learning." *International Conference on Machine Learning.* PMLR, 2023. [3] Clavier, Pierre, Tom Huix, and Alain Durmus. "VITS: Variational Inference Thompson Sampling for contextual bandits." arXiv preprint arXiv:2307.10167 (2023).