Learning Adversarial Low-rank Markov Decision Processes with Unknown Transition and Full-information Feedback

In this work, we study the low-rank MDPs with adversarially changed losses in the full-information feedback setting. In particular, the unknown transition probability kernel admits a low-rank matrix decomposition \citep{REPUCB22}, and the loss functions may change adversarially but are revealed to the learner at the end of each episode. We propose a policy optimization-based algorithm POLO, and we prove that it attains the $\widetilde{O}(K^{\frac{5}{6}}A^{\frac{1}{2}}d\ln(1+M)/(1-\gamma)^2)$ regret guarantee, where $d$ is rank of the transition kernel (and hence the dimension of the unknown representations), $A$ is the cardinality of the action space, $M$ is the cardinality of the model class, and $\gamma$ is the discounted factor. Notably, our algorithm is oracle-efficient and has a regret guarantee with no dependence on the size of potentially arbitrarily large state space. Furthermore, we also prove an $\Omega(\frac{\gamma^2}{1-\gamma} \sqrt{d A K})$ regret lower bound for this problem, showing that low-rank MDPs are statistically more difficult to learn than linear MDPs in the regret minimization setting. To the best of our knowledge, we present the first algorithm that interleaves representation learning, exploration, and exploitation to achieve the sublinear regret guarantee for RL with nonlinear function approximation and adversarial losses.

Paper

Similar papers

Peer review

Reviewer aWbZ6/10 · confidence 3/52023-07-02

Summary

The paper studies low-rank MDPs with adversarially changing losses in the full-information feedback setting. They assume the unknown transition probability function admits a low-rank matrix decomposition. They present POLO algorithm, a policy optimization-based algorithm, and prove it has sublinear regret in the number of episodes $K$, but linear dependency in $d$ which is the rank of the transition kernel. The regret is w.r.t the optimal policy for the discounted return value function. The authors claim they are the first to present and algorithm that interleaves representation learning, exploration, and exploitation to achieve the sublinear regret guarantee for RL with nonlinear function approximation and adversarial losses.

Strengths

1. POLO is the first algorithm that achieves sublinear regret with unknown features (although the setting is full information). 2. The authors provide a clear comparison between their work to previous works in the area, which emphasis their contribution. 3. Regret upper bound stated clearly, the analysis sketch is standard and clear (not fully checked).

Weaknesses

1. The full information feedback is a restrictive and unrealistic assumption. 2. The extensive literature review in Section 2 can be shorter, and instead use that space to have more proof sketches. That, in my opinion, can benefit the reader.

Questions

1. What is the difference between low-rank MDPs to Generalized Linear Model? As I see it, GLM is more general model. Do your results hold for this setting as well? 2. Same question for low Bellman rank MDPs, that also seem to generalize low-rank MDPs. 3. Do you have thoughts about how to remove the full-information assumption?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

n/a

Reviewer ci4p6/10 · confidence 3/52023-07-04

Summary

This work focuses on the low-rank MDPs with adversarial losses in the full-information feedback setting. Different from the previous work which assumes known features, this work considers the combination of representation learning and regret minimization problem, and is of the first result under this specific topic. The low-rank MDPs are defined in Definition 3.1, which admit certain low-rank feature structure that the transition dynamics can be presented with respect to two feature embedding functions. As described in the interaction protocol, the learner does not have accesses to the feature functions, and is required to learn these functions via the interaction between the environment. While the learner has to collect enough samples to estimate the feature functions, the goal of the learner is to minimize her regret (defined in Line 201), that is, the difference between the cumulative loss suffered, and that of the optimal policy in hindsight. The loss functions are selected adversarially that can change from episode to episode arbitrarily. To tackle these two problems simultaneously, the authors combine the techniques of representation learning and the RL with linear MDPs and adversarial losses together into Algorithm 1. Line 5-10 follow the idea of Uehara et al. [2022] to collect the samples for representation learning, Line 11-17 follow the idea of learning adversarial Linear MDPs like the work of Cai et al. [2020], given the estimated feature functions solved in Eq. (1). The result is presented in Theorem 5.1, which guarantees that the regret of Algorithm 1 is bounded by $\widetilde{\mathcal{O}}( \frac{K^{3/4}}{1-\gamma} + \frac{\sqrt{K}}{(1-\gamma)^2})$, where the first term is attributed to the representation learning of unknown features, and the second term is for the adversarial losses.

Strengths

1. Prior to this work, it is unknown whether the regret minimization problem and the representation learning problem can be dealt simultaneously with adversarial losses. This work is the first one to achieve sub-linear regret, which sheds light on this direction. 2. Based on the previous works on the representation learning of MDPs (such as Uehara et al. [2022] and Agarwal et al. [2020]), and on the linear MDPs (or low-rank MDPs) with adversarial losses (such as Cai et al. [2020] and Luo et al. [2021a]), the authors combine the techniques of these two fields hierarchically, under a novel scheme `doubled exploration and exploitation', to address the regret minimization problem.

Weaknesses

1. The designed algorithm suffers from huge computation cost (Line 13, 15, 16, and 17) when the state space is very large, which is the same as the previous algorithm OPPO in the previous work of Cai et al. [2020]. It is not very clear whether this could be improved with any planner oracle.

Questions

1. This result $\mathcal{O}(K^{3/4})$ does not match the $\sqrt{K}$ regret lower bound for the adversarial losses. Any specific conjecture on the lower bound/upper bound? 2. Any conjecture on the bandit feedback setting when limited the MDPs to the linear Mixture MDPS of Zhao et al. [2023]? 3. The loss function $\ell_k$ is designed as a function of $ S \times A \rightarrow [0,1]$, which does not admit the linear structure. Any idea to improve the result given the fact that $\ell_k(s,a) = \theta_k^\top \phi^\star(s,a) $ when $\theta_k$ is revealed at the end of the episode $k$? 4. The second term in the regret bound, comparing with the regret bound of OPPO and LSUOB-REPS, shaves a factor of $d$? Could you explain why this holds true, and whether this implies that your algorithm achieves better performance given the feature functions?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

This work is pure theoretical, does not have any potential negative societal impact.

Reviewer n9xa7/10 · confidence 3/52023-07-06

Summary

The authors consider the problem of learning an adversarial low-rank infinite episodic MDP with unknown transition and full information feedback. The idea behind this problem is that in many RL applications, the state and action spaces might be prohibitively large, rending results that scale with these metrics meaningless. Several approaches have used the assumption that there is a feature mapping state-action pairs to a low dimensional embedding space. While the common practice is to assume that this embedding is known, in the present problem the learner has to learn how the data is represented. The second challenge faced by the learner is that the loss functions change adversarially. These two challenges have been studied separately, but this work is the first to study them simultaneously. To do so, they work under the assumption that there exists an unknown low-rank embedding of the state-action pairs and propose an algorithm to tackle this problem: Policy Optimization for LOw rank MDPS (POLO). This algorithm relies on the standard Online Mirror Descent technique, combined with a bonus function which lowers the observed losses to favorize exploration. The authors then provide a rigourous analysis of the regret and show that their algorithm reaches sublinear regret. The regret analysis follows a standard decomposition of the regret into an estimation bias term, the OMD regret term and an optimism term, which respectively compare the true value function with the eztimated value function of the policy played by the learner, the difference between the performance of the policy played by the learner and the best policy in hindsight on the estimated value function, and finally the difference between the estimated value function and the true value function for the best policy.

Strengths

This paper tackles a new and challenging problem by combining two notably difficult aspects of online learning with MDPs: handling adversarial losses and having to learn the state-action mapping. Each of these problems has received a lot of attention in the past few years and are really relevant for the online learning community. The authors achieve their goal by presenting an algorithm that adapts to both of these aspects simultaneously and obtains sublinear regret. The algorithm that they present buil;dss upon standard tools in the litterature and has an appreciably clear and compact analysis. The authors properly discuss the limitations of their work, notably by questioning the computational efficiency of their approach and ensuring that their algorithm is coomputationally feasible. Overall, this paper will provide a good baseline for the problems of representation learning of MDPs in adversarial environments.

Weaknesses

It would be interesting to provide a the lower bound for the problem: from the simple online learning with full information feedback, having at least $\sqrt{K}$ regret from playing adversarial losses is unavoidable. In this work, it is stated that the first term in the regret bound, which scales with $K^{3/4}/(1 - \gamma)$ comes from learning the unknown transition kernel $P^\star$. Do you have a lower bound for this part the representation learning part of the problem?

Questions

Is there a lower bound for this problem, and could you elaborate on the cost of learning the representation?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors provided an interesting discussion of the potential limitations of their work and addressed them adequately.

Reviewer qWwQ5/10 · confidence 3/52023-07-10

Summary

This work studies low-rank MDPs with unknown and fixed transition and full-information adversarial losses. The proposed algorithm generalize RepUCB from the fixed reward setting to the adversarial reward setting. The main idea of the algorithm is to replace the greedy policy in RepUCB to an incremental policy update with exponential weights. Also, the proposed algorithm interleaves exploration and exploitation to obtain low regret.

Strengths

- This is an elegant generalization of RepUCB to the adversarial setting. It is also the first paper that simultaneously deals with non-linear function approximation and adversarial losses.

Weaknesses

- The technical novelty is a bit limited -- Given OPPO and POWERS, it becomes more or less clear that the key technique for unknown transition + full-information loss feedback setting is to ensure optimism and use a no-regret algorithm over the optimistically-biased losses. Though the form of optimism in low-rank MDP (i.e., optimism only in the initial state) is slightly different from those in linear mixture MDP, with full-information loss feedback, the key challenges is already resolved in RepUCB. It is not surprising that most of the analysis follows those of RepUCB except for the OMD regret term. - The proposed algorithm performs an epsilon-greedy styled exploration, which leads to a sub-optimal $K^{3/4}$ regret bound. To improve the technical novelty, I think the paper should try to design some less trivial exploration scheme and improve over the $K^{3/4}$ bound. Another direction to improve the technical novelty is to study the bandit feedback case. - The comparison with Rep-UCB around Line 259-268 and the comment that your algorithm is tighter in $d, A, \gamma$ is slightly off. The reason is that your bound is better than RepUCB only when $K^{3/4}A^{1/2}d/(1-\gamma) < d^{4/3}A^{2/3}K^{2/3}/(1-\gamma)^{4/3}$, which is equivalent to $K<d^4A^2/(1-\gamma)^4$. In this regime, both regret bounds $K^{3/4}A^{1/2}d/(1-\gamma)$ and $d^{4/3}A^{2/3}K^{2/3}/(1-\gamma)^{4/3}$ are at least $K$, meaning that this is an *vacuous* regime where both regret bounds are meaningless. In other words, there is no case your regret bound is better than that of RepUCB. If we calculate the sample complexity, the two algorithms would give $\frac{A^2d^4}{(1-\gamma)^4\epsilon^4}$ and $\frac{A^2d^4}{(1-\gamma)^4\epsilon^3}$ respectively, so there is actually no improvement in terms of $d, A, (1-\gamma)$.

Questions

See the above sections.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

There is no societal limitation.

Reviewer aWbZ2023-08-10

I thank the authors for their response and have no further questions.

Authorsrebuttal2023-08-15

Rewriting the Remaining Proof (Part 1)

Since some equations were not successfully rendered, we now rewrite the proof of this part. **Proof of the Regret Lower Bound (Cont.)** and by $\mathbb{E}\_{\left(i^*, a^*\right)} \triangleq \mathbb{E}\_{\operatorname{Alg}, \mathcal{M}\_{\left(i^*, a^*\right)}}$ the expectation with respect to $\mathbb{P}\_{\left(i^*, a^*\right)}$. **Step 1: Regret of $\operatorname{Alg}$ in $\mathcal{M}\_{(i^\ast,a^\ast)}$** For some $\mathcal{M}\_{(i^\ast,a^\ast)}$, its optimal policy $\pi^\ast_{(i^\ast,a^\ast)}:\mathcal{S}\to\mathcal{A}$ satisfies that $\pi^\ast\_{(i^\ast,a^\ast)}(s_{1,1})=a\_{i^\ast}$ and $\pi^\ast\_{(i^\ast,a^\ast)}(s\_{2,i^\ast})=a^\ast$, with the optimal value function $$ \begin{align} V\_{0}^*(s\_{1,1}) & =\mathbb{E}\left[\sum_{\tau=0}^{+\infty} \gamma^\tau r(s\_{\tau}, a\_\tau) \mid \pi^\ast\_{(i^\ast,a^\ast)},P^\star\_{(i^\ast,a^\ast)}, s\_0=s\_{1,1}\right]=\sum\_{\tau=2}^{+\infty} \gamma^\tau \left(\frac{1}{2}+\varepsilon\right)=\frac{\gamma^2}{1-\gamma}\left(\frac{1}{2}+\varepsilon\right).\tag{1} \end{align} $$ For some policy $\pi$, it is also clear that its value function satisfies $$ \begin{align} V_{0}^\pi(s\_{1,1}) &=\frac{\gamma^2}{1-\gamma}\left(\frac{1}{2}+\varepsilon\mathbb{P}\_{\left(i^*, a^*\right)}\left((s\_2,a\_2)=(s\_{2,i^\ast},a^\ast)\right)\right).\tag{2} \end{align} $$ Combining Eq. $(1)$ and Eq. $(2)$ shows that the regret of $\operatorname{Alg}$ in $\mathcal{M}\_{(i^\ast,a^\ast)}$ satisfies $$ \begin{align*} R\_K(\operatorname{Alg}, \mathcal{M}\_{(i^\ast,a^\ast}))&=\frac{\gamma^2\varepsilon}{1-\gamma}K\left(1-\frac{1}{K}\mathbb{E}\_{(i^\ast,a^\ast)}\left[\sum_{k=1}^K\mathbb{I}\{(s^k\_2,a^k\_2)=(s\_{2,i^\ast},a^\ast)\} \right]\right)\\ &=\frac{\gamma^2\varepsilon}{1-\gamma}K\left(1-\frac{1}{K}\mathbb{E}\_{(i^\ast,a^\ast)}\left[N^K\_{(i^\ast,a^\ast)} \right]\right)\notag, \end{align*} $$ where we define $N^K\_{(i^\ast,a^\ast)}= \sum\_{k=1}^K\mathbb{I}\\{(s^k\_2,a^k\_2)=(s\_{2,i^\ast},a^\ast)\\}$. **Step 2: Maximum regret of $\operatorname{Alg}$ over all possible $\mathcal{M}\_{(i^\ast,a^\ast)}$** With $R\_K(\operatorname{Alg}, \mathcal{M}\_{(i^\ast,a^\ast}))$ in the above equation, we can deduce that $$ \begin{align} \max\_{(i^\ast,a^\ast)}R\_K(\operatorname{Alg},\mathcal{M}\_{(i^\ast,a^\ast)})&\geq \frac{1}{(d-4)A}\sum\_{(i^\ast,a^\ast)}R\_K(\operatorname{Alg},\mathcal{M}\_{(i^\ast,a^\ast}))\notag\\ &\geq \frac{\gamma^2\varepsilon}{1-\gamma}K\left(1-\frac{1}{K(d-4)A}\sum\_{(i^\ast,a^\ast)}\mathbb{E}\_{(i^\ast,a^\ast)}\left[N^K\_{(i^\ast,a^\ast)} \right]\right).\tag{3} \end{align} $$ To lower bound the above display, it remains to upper bound $\sum\_{(i^\ast,a^\ast)}\mathbb{E}\_{(i^\ast,a^\ast)}\left[N^K\_{(i^\ast,a^\ast)} \right]$. To this end, by Lemma 1 in the work of [4] together with the fact that $N^K\_{(i^\ast,a^\ast)}/K\in[0,1]$, it holds that $$ \begin{align*} \operatorname{KL}\left(\operatorname{Ber}\left(\frac{1}{K}\mathbb{E}\_0\left[N^K\_{(i^\ast,a^\ast)}\right]\right),\operatorname{Ber}\left(\frac{1}{K}\mathbb{E}\_{(i^\ast,a^\ast)}\left[N^K\_{(i^\ast,a^\ast)}\right]\right)\right)\leq \operatorname{KL}\left(\mathbb{P}\_0,\mathbb{P}\_{(i^\ast,a^\ast)}\right). \end{align*} $$ This implies that $$ \begin{align*} \frac{1}{K}\mathbb{E}\_{(i^\ast,a^\ast)}\left[N^K\_{(i^\ast,a^\ast)}\right] &\leq \frac{1}{K}\mathbb{E}\_0 \left[N^K\_{(i^\ast,a^\ast)}\right]+\sqrt{\frac{1}{2}\operatorname{KL}\left(\mathbb{P}\_0,\mathbb{P}\_{(i^\ast,a^\ast)}\right)}\\ &=\frac{1}{K}\mathbb{E}_0 \left[N^K\_{(i^\ast,a^\ast)}\right]+\varepsilon\sqrt{2}\sqrt{\mathbb{E}\_0\left[N^K\_{(i^\ast,a^\ast)}\right]}, \end{align*} $$ where the inequality is due to Pinsker’s inequality that $(p-q)^2 \leq \frac{1}{2} \operatorname{KL}(\operatorname{Ber}(p), \operatorname{Ber}(q))$, for $p,q\in[0,1]$, and the equality comes from Lemma 15.1 of [1] and Lemma 14 of [2] as well as assuming $0\leq\varepsilon\leq\frac{1}{4}$. Based on this, one can see that $$ \begin{align} \frac{1}{K}\sum\_{(i^\ast,a^\ast)}\mathbb{E}\_{(i^\ast,a^\ast)}\left[N^K\_{(i^\ast,a^\ast)}\right] &\leq \frac{1}{K}\sum\_{(i^\ast,a^\ast)}\mathbb{E}\_0 \left[N^K\_{(i^\ast,a^\ast)}\right]+\varepsilon\sqrt{2}\sum\_{(i^\ast,a^\ast)}\sqrt{\mathbb{E}\_0\left[N^K\_{(i^\ast,a^\ast)}\right]}\notag\\ &\leq 1+\varepsilon\sqrt{2}\sqrt{(d-4)AK},\tag{4} \end{align} $$ where the second inequality follows from using the Cauchy-Schwartz inequality together with the fact that $N^K\_{(i^\ast,a^\ast)}\leq K$.

Authorsrebuttal2023-08-15

Rewriting the Remaining Proof (Part 2)

**Step 3: Optimizing $\varepsilon$ to lower bound the maximum regret** Substituting Eq. $(4)$ into Eq. $(3)$ leads to $$ \begin{align*} \max\_{(i^\ast,a^\ast)}R\_K(\operatorname{Alg},\mathcal{M}\_{(i^\ast,a^\ast)}) &\geq\frac{\gamma^2\varepsilon}{1-\gamma}K\left(1-\frac{1}{(d-4)A}-\varepsilon\sqrt{2}\sqrt{\frac{K}{(d-4)A}}\right)\\ &\geq\frac{1}{4\sqrt{2}}\cdot\frac{\gamma^2}{1-\gamma}\left(1-\frac{1}{(d-4)A}\right)^2\sqrt{(d-4)AK}\\ &\geq \frac{361}{1600\sqrt{2}}\cdot\frac{\gamma^2}{1-\gamma}\sqrt{(d-4)AK}, \end{align*} $$ where the second inequality comes from by choosing $\varepsilon=\frac{1}{2\sqrt{2}}\left(1-\frac{1}{(d-4)A}\right)\sqrt{\frac{(d-4)A}{K}}$ and the last inequality is due to $d\geq 8$ and $A\geq d-3$. Finally, note that $\varepsilon\leq \frac{1}{4}$ is guaranteed when $K\geq 2(d-4)A$. The proof is thus concluded. **Q.E.D.** [1] Lattimore et al. Bandit algorithms. Cambridge University Press, 20. [2] Domingues et al. Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisited. ALT, 21. [3] Cheng et al. Improved Sample Complexity for Reward-free Reinforcement Learning under Low-rank MDPs. ICLR, 23. [4] Garivier et al. Explore first, exploit next: The true shape of regret in bandit problems. Mathematics of Operations Research, 19. [5] He et al. Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes. ICML, 23.

Authorsrebuttal2023-08-15

About the equations that were not successfully rendered

We now rewrite the sentences containing the equations in the proof of Theorem 1 that were not successfully rendered: * where $\mathcal{S}\_{\mathcal{O}}=\\{s\_{o,i}\\}_{i=1}^{S-d}$ denotes the set of 'outlier states', * we denote by $\mathbb{P}\_{\left(i^*, a^*\right)} \triangleq \mathbb{P}\_{\operatorname{Alg}, \mathcal{M}\_{\left(i^*, a^*\right)}}$ the probability measure over the outcomes induced by the interaction between $\operatorname{Alg}$ and $\mathcal{M}_{\left(i^*, a^*\right)}$,

Reviewer qWwQ2023-08-16

I agree that since the setting is not studied before, any result would be beneficial for future study, even though it's not close to optimal. I have raised my score.

Reviewer ci4p2023-08-17

Thank the authors for their response

Thank the authors for their response. I would like to keep my score. Good work!

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC