I first thank the author's detailed response.
**Problem-dependent parameters and $T$.** I understand some existing works require these conditions. However, doing research needs to explore new possibilities instead of sticking to old ways. More importantly, as far as I know, at least two different ways can partially remove them at least in the non-smooth and convex case.
1. Parameter-free algorithms from the online learning community: For example, see [1], which can remove the dependence on $D_1$. However, as mentioned by the authors, I agree this may require additional algorithmic modifications.
2. Recent advances in stochastic optimization: Another simpler way is to combine the DoG technique proposed by [2]. For example, see [3], which not only removes $D_1$ but also removes $T$ in a simpler way.
Hence, I highly encourage the authors to consider how to remove these parameters (even partially), which can significantly improve the quality of the paper.
In addition, the authors mentioned that the results can be easily adapted to settings where only the upper bounds of problem-dependent quantities are known. However, I am not very convinced since it is hard to imagine that this can be guaranteed in a streaming setting. Could you provide some concrete examples?
**Extra $\log\log T$.**
1. I clarify that I didn't claim this is a major weakness. Instead, I put it in the weakness only because it is undesired and suboptimal. I believe the final goal should always be to find the optimal bound.
2. Moreover, I also would like to point out that the optimal rate without any extra logarithmic has already been achieved in prior works like [4] (which is even mentioned by the authors) and [3]. But I understand the double $\log$ factor here may appear due to different reasons, which could require different ways to remove. However, as discussed above, this is still a weakness in my opinion.
3. I appreciate the authors' discussion on this weakness, which I recommend adding as a remark in the paper.
**References**
[1] Zhang, J., & Cutkosky, A. (2022). Parameter-free regret in high probability with heavy tails. Advances in Neural Information Processing Systems, 35, 8000-8012.
[2] Ivgi, M., Hinder, O., & Carmon, Y. (2023, July). Dog is sgd’s best friend: A parameter-free dynamic step size schedule. In International Conference on Machine Learning (pp. 14465-14499). PMLR.
[3] Liu, Z., & Zhou, Z. (2023). Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation. arXiv preprint arXiv:2303.12277.
[4] Nguyen, T. D., Nguyen, T. H., Ene, A., & Nguyen, H. (2023). Improved convergence in high probability of clipped gradient methods with heavy tailed noise. Advances in Neural Information Processing Systems, 36, 24191-24222.