We study the evaluation of a policy under best- and worst-case perturbations to a Markov decision process (MDP), using transition observations from the original MDP, whether they are generated under the same or a different policy. This is an important problem when there is the possibility of a shift between historical and future environments, $\textit{e.g.}$ due to unmeasured confounding, distributional shift, or an adversarial environment. We propose a perturbation model that allows changes in the transition kernel densities up to a given multiplicative factor or its reciprocal, extending the classic marginal sensitivity model (MSM) for single time-step decision-making to infinite-horizon RL. We characterize the sharp bounds on policy value under this model $\unicode{x2013}$ $\textit{i.e.}$, the tightest possible bounds based on transition observations from the original MDP $\unicode{x2013}$ and we study the estimation of these bounds from such transition observations. We develop an estimator with several important guarantees: it is semiparametrically efficient, and remains so even when certain necessary nuisance functions, such as worst-case Q-functions, are estimated at slow, nonparametric rates. Our estimator is also asymptotically normal, enabling straightforward statistical inference using Wald confidence intervals. Moreover, when certain nuisances are estimated inconsistently, the estimator still provides valid, albeit possibly not sharp, bounds on the policy value. We validate these properties in numerical simulations. The combination of accounting for environment shifts from train to test (robustness), being insensitive to nuisance-function estimation (orthogonality), and addressing the challenge of learning from finite samples (inference) together leads to credible and reliable policy evaluation.
Paper
Similar papers
Peer review
Summary
This paper proposes a statistical theory for off-policy evaluation given observational data from an unknown transition kernel of an original MDP, possibly under policies the same as or different from the target policy. This paper focuses on the situation of environment shift, i.e., the transition kernel of the MDPs at test time can shift from the original MDP to a certain extent. These transition kernels are assumed to be center around the observational transition kernel. This paper proposes new algorithms and sharp estimators for robust estimation of policy value. The proposed estimation method has several desired properties.. First, it is semiparametrically efficient, with a $\sqrt{n}$-rate of estimation even when certain nuisance functions are estimated at slow parametric rates. The estimator is also asymptotically normal which enables easy statistical inference. Even when the nuisance functions are estimated with bias, the estimator is still constant. Numerical simulations verify the efficiency and sharpness of the proposed estimator.
Strengths
**Significance**: **1.** This paper focuses on an important and challenging theoretical problem of policy evaluation under transition kernel shift. This is a very realistic and probably more common scenario compared to the classical OPE setting where only the policies shift. **2** This paper presents a credible statistical theory for policy evaluation under transition kernel shift and the existence of nuisance functions. The statistical analysis shows that strong statistical guarantee can be achieved for the proposed estimator. **Quality**: this paper has a good quality. **1.** The paper has a solid theoretical analysis, with well specified definitions and theorems.
Weaknesses
**1.** This paper is purely theoretical and thus it is very hard to evaluate the practicability of the proposed estimation framework. The numerical simulation also only covers a very simplified synthetic environment. Thus it is unclear how useful or computationally efficient the proposed estimation framework really is. **2.** I feel this paper is missing a discussion of the line of work on OPE under POMDP. It seems that the OPE on POMDP is naturally connected with the topic of this paper, though I am not very familiar with that line of work. **Minor**: **1.** The notation of this paper is a bit heavy. For example, the plus-minus sign confused me for a while when I read about its meaning (“read twice”) at the bottom of page 3. But this is not a big issue and is sort of understandable given the complex framework of this paper. **2.** This paper is a bit hard to read for readers unfamiliar with this topic. No example or demonstration of any kind is given.
Questions
Please see weakness.
Rating
6
Confidence
3
Soundness
3
Presentation
2
Contribution
2
Limitations
Yes.
Summary
This paper studies the problem of evaluating a policy under best- and worst-case perturbations to a Markov decision process (MDP), given transition observations from the original MDP. The first contribution is the robust Q learing algorithm for CVaR problem, the second contribution is the estimation of robust visitation distribution in the function approximation setting and their complexity bounds.
Strengths
The problem studies in this paper is of significant practical importance, which is to evaluate the worst/best performance over an uncertainty set. The authors designed algorithms and derived their complexity bounds, which are shown to be optimal.
Weaknesses
- The authors seem to be using offline and off-policy interchangably, which is misleading and confusing. - One important challenge in offline reinforcement learning is due to partial coverage and distribution shift. However, this paper does not explicitly define the single-policy concentrability coefficient that measures the gap between the sampling distribution $\nu$ and the target policy $\pi$, and therefore did not consider this key challenge in the offline setting. - The notations are unnecessarily complicated, and many are used without being defined. For example, in algorithm 1 line 6, the hat on top of the expectation, and the subscript of the expectation is not clear. - There are many technical assumptions that remain unjustified. Please justify each of the assumptions, and provide examples when those assumptions can be satisfied, and how to verify those assumptions in practice. - The perturbation model in eq. (1) is unjustified. It could happen easily that $\Lambda(s,a)$ is zero or infinite for some $(s,a)$. The authors shall explain why this perturbation model is of interest, and why this model is superior over existing uncertainty set models, e.g., those defined by KL divergence, TV.
Questions
- There are several other important papers on robust offline reinforcement learning that are missing in this paper, e.g., Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage Jose Blanchet, Miao Lu, Tong Zhang, Han Zhong Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity Laixi Shi, Yuejie Chi Comparison with existing robust offline RL is not clear in this paper. - The writing of this paper needs to be improved. Is Section 3 and Algorithm 1 only about CVaR?
Rating
6
Confidence
4
Soundness
3
Presentation
1
Contribution
2
Limitations
na
Response to Reviewer ghkS
Dear Reviewer ghkS, Thanks for specifying your questions and for taking the time to consider our response. We will now answer them in order: **W3 not clear from the rebuttal.** To clarify, the $\hat{\mathbb{E}}_D$ notation (where $D$ is a dataset of points $x_1, \dots, x_n$) is defined as follows. For any function $f(x)$, we define $\hat{\mathbb{E}}_D[f(x)]=\frac{1}{n}\sum_if(x_i)$. In other words, $\hat{\mathbb{E}}_D$ is the empirical expectation for the dataset $D$. In the revision, we will directly use the $\frac{1}{n}\sum_if(x_i)$ in Alg 1, Line 6 to avoid using this notation. **W4 the reviewer would like to see why those assumptions are reasonable assumptions and how can they be satisfied in practice.** We are happy to provide clarifications on why these assumptions are reasonable and when they are satisfied. We now discuss each assumption in the paper. **Assumption 3.2.** This assumption posits that the quantile regression step succeeds in Alg 1. Quantile regression is a well-established problem with many classical algorithms for doing so: for example, Meinshausen and Ridgeway showed that random forests can learn conditional quantiles and this satisfies Assumption 3.2 where the $err_{QR}$ converges to zero as the number of datapoints grows to infinity (Theorem 1 of Meinshausen and Ridgeway 2006). More recently, there are also effective deep RL approaches that successfully use neural networks to learn quantiles, such as QR-DQN (Dabney et al 2018a) and IQN (Dabney et al 2018b). These approaches have obtained state-of-the-art results in Atari and can also be used to satisfy this assumption. **Assumption 3.3.** The Bellman completeness (BC) assumption posits that the $Q$ function class is closed under the (robust) Bellman operator. BC is a standard assumption for proving convergence in offline RL (Xie et al. 2021) and has also appeared in prior robust MDP works (Panaganti et al 2022; Bruns-Smith and Zhou 2023). In fact, BC is necessary for TD-style algorithms to succeed; without it, TD can diverge or converge to bad fixed points, e.g. Tsitsiklis and Van Roy, 1996 showed such a counterexample. Since our Algorithm 1 is based on TD, it is quite natural for our results to also rely on BC. **Assumption 4.2.** This is a mild regularity assumption that simply requires (i) the outputs of our function class to have bounded outputs and (ii) the thresholding function $V^\pm(s')-\beta^\pm(s,a)$ to be continuously distributed around the point $0$. We note that our algorithms do not crucially rely on (ii); if the distribution is discrete, we can use the discrete form for CVaR. **Assumption 4.3.** This assumption is a realizability assumption for importance-weighted estimators (i.e. Algorithm 2). The first part (i) simply requires that our density function class $\mathcal{W}$ to realize the density ratio and can be satisfied as we increase our function class's expressivity. The second part (ii) posits that our adversary function class $\mathcal{F}$ is rich enough to capture the error terms that take the form $\mathcal{J}'(w-w^\pm)$ for $w\in\mathcal{W}$. We note that both (i) and (ii) are *monotone* in the function class size and can be satisfied by making functions more expressive (i.e. increasing size of neural network). Regarding Assumption 4.3, we remark that our algorithms and theory are in fact robust to violations. In particular, we proved in Appendix I that the error from Algorithm 2 will depend gracefully on the approximation errors to Assumption 4.3 (as we formalized in Assumption I.1). In fact, this robustness also holds for the BC assumption (Assumption 3.3), i.e. our bounds will gracefully degrade with the robust Bellman error: $\epsilon_{be} = \min_{q\in\mathcal{Q}}\max_{q'\in\mathcal{Q}}\|\|q-\mathcal{T}^\pm_{rob}q'\|\|_2$. We will add the robustness-to-BC result in our revision. To conclude, our theory and algorithms are robust even when these assumptions fail to hold exactly. **W5 why it ($\Lambda$) cannot be zero or infinity?** Recall that $\Lambda$ is a parameter chosen by the user to capture the magnitude of perturbations $U$, which ought to satisfy $\Lambda^{-1}(s,a) \leq \frac{dU(s'\mid s,a)}{dP(s'\mid s,a)} \leq \Lambda(s,a)$. Since the definition implies $\Lambda^{-1}(s,a)\leq\Lambda(s,a)$, we have that $\Lambda(s,a)\geq 1$ so it cannot be $0$ and must be at least $1$. The infinity case is possible: if $\Lambda(s,a)=\infty$, then $U(s'\mid s,a)$ is allowed to be arbitrarily different from $P(s'\mid s,a)$, and our robust evaluation would evaluate the best- or worst-paths, i.e. what is the maximum or minimum possible cumulative reward. This setting was also studied by Du et al. 2023. Thank you again so much for the detailed feedback and for taking our responses into consideration. Please let us know if you have more questions and we will do our best to answer them promptly. Thank you again for your valuable efforts in reviewing our paper.
More Citations
**Citations** [1] Du, Yihan, Siwei Wang, and Longbo Huang. "Provably efficient risk-sensitive reinforcement learning: Iterated cvar and worst path." ICLR 2023. [2] Meinshausen, Nicolai, and Greg Ridgeway. "Quantile regression forests." Journal of machine learning research 7.6 (2006). [3] Dabney, Will, et al. "Distributional reinforcement learning with quantile regression." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018a. [4] Dabney, Will, et al. "Implicit quantile networks for distributional reinforcement learning." International conference on machine learning. PMLR, 2018b. [5] Panaganti, Kishan, et al. "Robust reinforcement learning using offline data." Advances in neural information processing systems 35 (2022): 32211-32224. [6] Tsitsiklis, John, and Benjamin Van Roy. "Analysis of temporal-diffference learning with function approximation." Advances in neural information processing systems 9 (1996). [7] David Bruns-Smith and Angela Zhou. Robust fitted-q-evaluation and iteration under sequentially exogenous unobserved confounders. arXiv preprint arXiv:2302.00662, 2023. [8] Xie, Tengyang, et al. "Bellman-consistent pessimism for offline reinforcement learning." Advances in neural information processing systems 34 (2021): 6683-6694.
Dear reviewer, the open discussion period is nearing the end. We hope our responses addressed your concerns. If you feel that they have we would greatly appreciate if you update your score accordingly. If you still have questions let us know. We worked hard to be thorough while clear in our response and address all questions raised. Thank you again!
Thank you for the response! My concerns are addressed, and therefore, I raised the score to 6.
Summary
This paper studies off-policy evaluation problems in robust Markov decision processes, where the environment dynamics may change over time. The authors consider a perturbation model that modifies the transition probability up to a given multiplicative factor. They first propose two new algorithms for learning the Q-value function $Q$ and the density ratio function $w$. Then, they design a new policy value estimator based on the estimated nuisance parameters. The authors provide a theoretical analysis of the new algorithms and estimators, along with numerical experimental results to verify their effectiveness.
Strengths
a. The studied problem of off-policy evaluation in robust MDPs is interesting and meaningful. b. The theoretical analysis is solid. The proposed $Q$-function estimation algorithm achieves a $\widetilde{\mathcal{O}}(n^{-1/2})$ rate for parametric classes. More importantly, the proposed orthogonal estimator in Section 5 has a second-order dependence on the estimation error of nuisance parameters. As a result, it only requires the rate of nuisance parameter estimation to be faster than $n^{-1/4}$ to achieve an $n^{-1/2}$ rate. c. The numerical results align with the theoretical analysis; the orthogonal estimator consistently achieves the best performance compared to the $Q$ estimator and $w$ estimator.
Weaknesses
a. This paper considers a perturbation model where the ratio between the modified transition kernel and the nominal kernel is bounded by a sensitivity parameter. In Algorithms 1 and 2, the learner needs access to the parameter $\Lambda(s,a)$. However, it is unclear how to estimate $\Lambda(s,a)$ in real-world scenarios. Additionally, the authors claim that the perturbation model is more general and captures confounded settings. More detailed analysis and explanations are needed to support this claim. b. Algorithm 2 requires solving a minimax optimization problem, which is usually very time-consuming in practice. In the experiments, the authors use an iterative method as an approximation. Is there any theoretical analysis for that implementation? c. Additional experiments using real-world datasets would help to better demonstrate the practical applicability of the proposed algorithms. Minor: Redundant content in line 194
Questions
See weakness part.
Rating
6
Confidence
3
Soundness
3
Presentation
3
Contribution
3
Limitations
N/A
Summary
This paper studies statistically-efficient robust/optimistic (e.g., worst- or best-case within the uncertainty set $\mathcal{U}$) off-policy evaluation with bounded Radon-Nikodym uncertainty sets centered around known nominal models. It proposes novel algorithms for computing robust/optimistic Q functions, and provides theoretical analysis for them. It further claims that the sharp and efficient estimators are optimal in the local-minimax sense, and provides preliminary experimental results to support the claim.
Strengths
* The paper seems to include a wide collection of new theoretical results, with assumptions clearly stated and proofs attached. * The problem itself is simple, fundamental, and stated in an comprehensible way.
Weaknesses
* The writing of this paper needs to be improved, at least for non-experts in this specific area. * Frankly speaking, I'm having a lot of trouble following the paper, or even understand the problems and the results. Though I have to admit that I major in CS and don't have a strong statistics background. * First and foremost, it doesn't seem like the $\pm$ superscript is doing any good here, except confusing the reader for the first few moments. If I were to write this paper, I would just sell the story of "robust OPE", and say a sentence or two on how to re-interpret it as a optimistic evaluation scheme (I do think negating rewards is easier to understand). * Some sentences are broken and not fully comprehensible (e.g., lines 193~195), some notations are left undefined before their first use (e.g., the $\frac{dU}{dP}$ notation, which I regard as Radon-Nikodym derivative), and some paragraphs are duplicated with subtle differences (e.g., Lemma 4.1 and G.1). * The literature review part (Section 1.1 and Appendix B) seems to be very incomplete, and some paragraphs seem irrelevant (e.g., lines 84~104). Since this is a paper on robust OPE, a more comprehensive review on existing works regarding robust MDPs and OPE methods is definitely needed, especially a comparison on the settings and the bounds. * As far as I'm concerned, the flow of the whole paper needs major revision. For example, it's unclear why we need to estimate $w^{\pm}$ in Section 4, until in the estimator for $V^{\pm}\_{d\_1}$ is proposed in Section 5. Another issue is that I don't fully get the point why we need a few more complex procedures to estimate $V^{\pm}\_{d\_1}$ after we've got the estimates for $Q^{\pm}$. Besides, the key results are not explained in plain English (e.g., given the sentences between lines 211 and 217, I still don't see clearly what Theorem 3.4 means. What is the typical order of $\varepsilon^{\mathcal{Q}}_n$? What about the $\mathrm{err}$ term?) * Simply put, Sections 3~5 are not well-motivated, the results are not intuitively interpreted, and their connections need to be further specified. * Personally, I expect to first see a full overview of the proposed algorithm and the "ultimate" end-to-end sample complexity result (which is now hidden in the paragraph "Putting everything together" on lines 316~322) to get the big picture, before diving into all the technical details introducing the subroutines. * Since I'm not an expert in this area, I don't fully understand the technical details. (***To AC:*** Please do take other reviewers' opinions for technical soundness. I've tried hard but failed to fully check the proofs.) * Specifically, I don't follow the proofs in Sections E.5 (for the second part of Theorem 5.3), G.1 (for Lemma 3.1) and G.2 (for Lemma 4.1). The latter two don't seem like legit proofs in my opinion.
Questions
* Why do you use bounded Radon-Nikodym derivative for defining ambiguity sets, as I haven't seen people doing this? Does it have any connections with other "distances", e.g. total variation, KL-divergence, or Wasserstein distance?
Rating
4
Confidence
2
Soundness
2
Presentation
1
Contribution
3
Limitations
There are a few sentences discussing limitations of the setting, but a summary is missing.
Thank you very much for specifying your questions about the technical appendix. We greatly appreciate the detailed review – this helps us improve our paper. We will answer them in order: * "The equation on line 657 seems to happen only with high probability" You are correct: this occurs with high probability, which is why we used the big-O in probability ($O_p$) notation in this expression. This in fact is simply a restatement of the display equation just above this line, arising from Chebychev's inequality. Recall that $a_n=O_p(b_n)$ iff for any $\eta>0$ we have $P(a_n/b_n\geq \epsilon)\leq\eta$ for some $\epsilon>0$. This is precisely what we get from the display by letting $\epsilon=1/\sqrt{\eta}$ with $a_n=\varepsilon_k$ and $b_n=\mathrm{Var}[\epsilon_k\mid \mathcal D_k)]^{1/2}$ and the probability $P$ being conditional on $\mathcal D_k$, $P(\cdot\mid\mathcal D_k)$. We will add a sentence here to make it extra clear. * "the following sentence on line 658 doesn't make any sense to me (What is $n_K$ Why does it matter here?)" Here we are simply defining $n_K$ in-line as $n/K$, being the size of the dataset $\mathcal D_k$, to make extra clear why the asymptotics are the same whether we take $n\to\infty$ or the size of the fold (wrt which we are taking Chebychev) to infinity. This is to clarify that the conditional big-O in probability ($O_p(\cdot\mid \mathcal D_k)$) indeed makes logical sense here. Again, we will add a sentence here to make it extra clear. * "In Appendix G.1, I don't know how to relate the equation between lines 717 and 718 to the statement of Lemma 3.1 (specifically, the form of $\mathcal{T}_{rob}$). Besides, there is a typo in the equation (nothing after $\sup$)." Firstly, thank you for catching the typo: the missing expression after the $\sup$ is $\mathbb E_{G}[f(s')]$. That this completes the proof follows by the definition of $G$ in line 715. In particular, inverting the definition, each feasible $U$ in the supremum in the definition of $\mathcal{T} _ {rob}$ can be equivalently expressed as $U = \Lambda^{-1}(s,a)P + (1-\Lambda^{-1}(s,a))G$ where $G$ satisfies the constraints in line 717. Thus, the supremum in $\mathcal{T} _ {rob}$ can be expressed as $\Lambda^{-1}(s,a)$ times the expectation under the nominal $P$, and $(1-\Lambda^{-1}(s,a))$ times the sup over $G$, i.e. $\sup_{U: \Lambda^{-1}(s,a)\leq \frac{dU(s'\mid s,a)}{dP(s'\mid s,a)}\leq \Lambda(s,a)}\mathbb{E} _ U[v(s')\mid s,a] = \Lambda^{-1}(s,a)\mathbb{E} _ P[v(s')\mid s,a] + (1-\Lambda^{-1}(s,a))\sup_{G\ll P: \|dG(\cdot\mid s,a)/dP(\cdot\mid s,a)\| _ \infty\leq \tau^{-1}(s,a)}\mathbb{E} _ G[v(s')]$. Finally, line 717 observes that the latter term is equivalent to CVaR, which is proved in Section 3.4 of [3]. Thank you for bringing up this question. Explaining the completion of the proof certainly merits another line or two -- although it follows from the previous lines, it can be made easier to follow. * "In Appendix G.2, two proof sketches are provided, but it's better to present only one in a clear and rigorous way." Thank you for the feedback. We hoped to provide intuition to the reader by providing two ways to think about this, but we do understand that a clear complete proof may be the most instructive. Here is a complete proof we can add here in the appendix: Fix any $s,a$ and let $\tau=\tau(s,a)$. Fix any function $v(s')\in\mathbb{R}$. We want to show that the worst-case $U^+ = \arg\max _ {U\in\mathcal{U}(P)} \mathbb{E} _ U[v(s')\mid s,a]$ has a closed form expression as shown in line 725. By the proof of Lemma 3.1 above, we can rewrite $U^+(s'\mid s,a) = \Lambda^{-1}(s,a)P(s'\mid s,a) + (1-\Lambda^{-1}(s,a)) G^+(s'\mid s,a)$, where $G^+ = \arg\max _ {G\ll P: \|dG(\cdot\mid s,a)/dP(\cdot\mid s,a)\| _ \infty\leq \tau^{-1}(s,a)}\mathbb{E} _ G[v(s')]$. Thus, it suffices to simplify $G^+$. To do so, we invoke the premise that the CDF of $v(s')$ is differentiable at $\beta^+ _ \tau$, i.e. $F _ v(\beta^+ _ {\tau,F _ v}(s,a)\mid s,a)=1-\tau$. This implies that the CVaR is exactly the conditional expectation of the $1-\tau(s,a)$-fraction of best outcomes, i.e. $\text{CVaR}^+ _ \tau(v(s')\mid s,a) = \mathbb{E}[v(s')\mid v(s')\geq\beta^+ _ \tau(s,a),s,a]$, which in turn is equal to $\tau^{-1}\mathbb{E}[v(s')\mathbb{I}[v(s')\geq\beta^+_\tau(s,a)]\mid s,a]$. Thus, $G^+(s'\mid s,a) = \tau^{-1}P(s'\mid s,a) \mathbb{I}[v(s')\geq\beta^+ _ \tau(s,a)]$. This concludes the proof for the $+$ case. The proof for the $-$ case is symmetric and follows identical steps. Thank you again so much for the detailed feedback about the technical appendix. **Citations** [1] Chernozhukov, Victor, et al. "Double/debiased machine learning for treatment and structural parameters." (2018): C1-C68. [2] Kallus, Nathan, and Masatoshi Uehara. "Double reinforcement learning for efficient off-policy evaluation in markov decision processes." Journal of Machine Learning Research 21.167 (2020): 1-63.
Dear reviewer, the open discussion period is nearing the end. We hope our responses addressed your concerns. If you feel that they have we would greatly appreciate if you update your score accordingly. If you still have questions let us know. We worked hard to be thorough while clear in our response and address all questions raised. Thank you again.
Thanks for the responses!
I appreciate the authors' efforts to settle my questions and concerns, especially their open attitude towards adjusting the presentation and flow of the paper. When I say I cannot follow the proofs, I'm not implying that the proofs are technically wrong or suspicious. Rather, I just cannot get through certain steps or sentences. For a few examples: * The equation on line 657 seems to happen only with high probability, and the following sentence on line 658 doesn't make any sense to me (What is $n_K$? Why does it matter here?) * In Appendix G.1, I don't know how to relate the equation between lines 717 and 718 to the statement of Lemma 3.1 (specifically, the form of $\mathcal{T}_{\mathrm{rob}}$. Besides, there is a typo in the equation (nothing after $\mathrm{sup}$). * In Appendix G.2, two proof sketches are provided, but it's better to present only one in a clear and rigorous way. I understand that I'm probably not the target audience of this paper (lol), and this is a tough (and kind of ignorant) decision, but I decide to keep my rating for now.
Thank you for the detailed response. W1 addressed W2 Addressed W3 not clear from the rebuttal W4 the reviewer would like to see why those assumptions are reasonable assumptions and how can they be satisfied in practice. W5 why it cannot be zero or infinity? Q1 and Q2 are addressed.
Thank you for your feedback. I will take all the responses into consideration during the discussion and decision session. In the paper revision, please make sure to (i) adding the above discussion in the paper revision to make the paper clearer and easier to read; (ii) adding results on real-world dataset, such as the health dataset mentioned in the rebuttal and perhaps other appropriate datasets. Best, reviewer
Thank you for the detailed response. I will maintain my positive score. However, I also suggest that the authors should make an effort to simplify the notation and enhance the readability of the paper.
Decision
Accept (poster)