Summary
The authors propose a time-varying extension of SAFEOPT to overcome the problems of time-varying rewards under time-varying safety constraints.
Under stationarity conditions, optimality guarantees are provided and the numerical simluation shows a (favorable) comparison to the SAFEOPT.
Strengths
1. The paper is very well written and easy to follow.
2. Based on related work, the problems of time-varying rewards under time-varying safety constraints are an open problem in literature, and his paper addresses that.
3. The paper provides formal safety guarantees for their TVSAFEOPT algorithm.
Weaknesses
1. *Some delineation to related work seems rather vague and requires stronger justification.* An example for TVSBO: the time-variable and temporal aspect of the kernel can just as well be interpreted as context using existing results. Perhaps a table would help here to highlight key aspects.
2. *Lack of real-world data experiments and comparison to related work.* To support the downsides of existing approaches, an empirical comparison to existing TVSBO approaches mentioned in the related work section would be needed.
3. The *empirical results could be more convincing* by adding a variety of initial safe sets and revised plots. The current plots/results are hard to parse.
4. It would be beneficial if the *theoretical/technical challenge of extending safety to the time-varying case were more detailed*. This would streamline the presentation and help in assessing the impact of the contribution.
Questions
1. *In the Appendix, a spatio-temporal SE kernel is introduced. How is this construction different from using an SE-ARD kernel with a composite variable $z = [x^T,t]^T$?* If I am not mistaken, for the SE-ARD kernel there would be no different than defining a single kernel with $z$.
2. It is mentioned that the Lipschitz constants are to be known beforehand. However, while commonly assumed, *how do you get a hold of an RKHS norm bound $B$ (related to Assumption 2.1) to compute the UCB?*
3. *Could you provide Figure 1 sooner in the manuscript?* It would be super helpful to see this central illustration already on page 2.
Limitations
1. Practicality of the safety guarantee: requiring many Lipschitz constants for both for space and time while also requiring an RKHS norm bound.
2. Theoretical and empirical impact: Lack of comparisons to TVSBO approaches makes the implications of the contribution unclear both theoretically and empirically.