Maximum State Entropy Exploration using Predecessor and Successor Representations

Animals have a developed ability to explore that aids them in important tasks such as locating food, exploring for shelter, and finding misplaced items. These exploration skills necessarily track where they have been so that they can plan for finding items with relative efficiency. Contemporary exploration algorithms often learn a less efficient exploration strategy because they either condition only on the current state or simply rely on making random open-loop exploratory moves. In this work, we propose $ηψ$-Learning, a method to learn efficient exploratory policies by conditioning on past episodic experience to make the next exploratory move. Specifically, $ηψ$-Learning learns an exploration policy that maximizes the entropy of the state visitation distribution of a single trajectory. Furthermore, we demonstrate how variants of the predecessor representation and successor representations can be combined to predict the state visitation entropy. Our experiments demonstrate the efficacy of $ηψ$-Learning to strategically explore the environment and maximize the state coverage with limited samples.

Paper

Similar papers

Peer review

Reviewer RqRN6/10 · confidence 4/52023-06-26

Summary

This paper addresses the problem of maximum state entropy exploration in environments without rewards. Especially, it proposes a method to learn an history-based policy maximizing the entropy over the states sampled in a single trajectory. The method, called $\eta\psi$-Learning, combines predecessor representations, to keep memory of the previously visited states, and successor representations, to predict the states that will be visited in expectation under the current policy. The introduced algorithm is tested against MaxEnt in a set of toy domains and some continuous control tasks, it is also briefly tested against VariBAD in gridwolrd domain.

Strengths

- The paper proposes a simple yet effective method to learn non-Markovian policies that maximize the state entropy induced by a single trajectory; - The introduced method remarkably improves the performance of MaxEnt, here taken as a representative algorithm for maximum state entropy exploration with Markovian policies; - The ideas are presented with clarity and with some nice visualizations (e.g., Fig. 1 and 3) to support the intuition.

Weaknesses

- It is unclear how the procedure can be scaled up to more challenging domains, e.g., image-based inputs; - The method is only tested against MaxEnt, which is not the state-of-the-art method for state entropy maximization, especially in continuous control tasks.

Questions

This is an overall interesting paper, tackling a relevant problem raised by (Mutti et al., 2022a) on how to learn non-Markovian policies for maximum state entropy exploration efficiently. Especially, the authors propose to circumvent the inherent computational intractability of the problem with a well-motivated function approximation approach, in which the current decision is conditioned on a compact representation of the history and a forecast of the future state visitations under the policy. Although the paper does not introduce particularly novel or surprising ideas, it presents the method with clarity and showcases a brilliant performance for $\eta\psi$-Learning, at least in comparison with MaxEnt. While those might be sufficient reasons for acceptance, I believe that including a discussion on how this procedure can be scaled up to more challenging domains, such as image-based tasks, would make for a significant leap in the value of the paper. *(Major. How to scale up the approach?)* The experiments show a remarkable improvement over MaxEnt, but previous works (e.g., Mutti et al., 2021) have demonstrated how Markovian policies maximizing the state entropy can be learned in challenging domains. Instead, it is unclear how the predecessor and successor representations of $\eta\psi$-Learning can be scaled to truly high-dimensional settings. While the method may be trivially adapted to continuous settings via discretization, it is unclear whether it can match the performance of previous works based on $k$-NN entropy estimators. Can the authors discuss avenues to scale their approach to high-dimensional domains? Estimating $\psi_\theta$ in practice would be significantly easier than state density estimation? *(Estimating the state visitations instead of the entropy)* In $\eta\psi$-Learning, the quality of an action is evaluated in terms of the future visitations it will induce under the policy, from which the entropy of the trajectory can be then computed. However, the task of estimating the future state visitations seems strictly harder w.r.t. what is really needed to make an optimal decision, i.e., an estimate of the entropy that future visitations will induce. I am wondering whether there is a way to directly target entropy estimation instead of future state visitations in $\eta\psi$-Learning. *(Experiments. MaxEnt performance)* The learning curve of MaxEnt is quite underwhelming: While the inferior performance of Markovian policies against $\eta\psi$-Learning is reasonable, I would have expected to see some learning progress for MaxEnt as well, especially in domains in which the random policy is far from the optimal strategy, such as in ChainMDP and RiverSwim. Can the authors clarify what is happening with the learning curve of MaxEnt? *(How the approach relates to Forward-Backward representations?)* A previous paper (Touati and Ollivier, Learning one representation to optimize all rewards, 2021) also addressed unsupervised reinforcement learning through representations of the past and future visitations, which they call forward-backward representations, bearing some similarity with the predecessor and successor representations of $\eta\psi$-Learning. Differently from this paper, they make learning these representations the actual unsupervised objective, and they advocate for directly employing forward-backward representations for zero-shot adaptation to the supervised task, instead of running state entropy maximizing policies first. Can the author compare $\eta\psi$-Learning with this previous approach, and explain whether the successor representation might be directly exploited in a supervised adaptation phase?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

The paper does not include an explicit assessment of the limitations of the presented approach, which would add significant value to the paper in my opinion.

Reviewer gBvc8/10 · confidence 4/52023-06-28

Summary

This paper shows a combination of "succesor" and "predecessor" representations can be used to develop an efficient maximum entropy exploration policy.

Strengths

- Overall, the paper is clearly written and makes a useful contribution to the exploration literature. - As far as I know, the ideas are novel. - The authors make a strong empirical case for their algorithm. - I'm a bit unsure about the significance of the algorithm, beyond the empirical results shown in the paper. I'm not sure whether the paper is sufficiently ground-breaking to have a significant impact on the broader reinforcement learning literature.

Weaknesses

- I think the authors are a bit loose with their arguments about human cognition. It is hotly debate to what extent cognition depends on language in a strong way. It also important to note that one can endorse a "language of thought" hypothesis about high-level cognition without endorsing the hypothesis that this corresponds to natural language. In any case, I appreciate that these points have little bearing on the substance of this paper. - Eq. 1 could benefit from more explanation. - Please state what error bars show in figures. UPDATE: the authors have addressed my comments.

Questions

The authors should be able to easily address my releatively minor comments in the "weaknesses" section. More broadly, I think the general usefulness of this approach will depend on how it can be applied beyond the maximum entropy exploration setting. UPDATE: the authors have addressed my comments.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

In the Discussion, the authors discuss several ways to improve and extend their algorithm.

Reviewer L5xE5/10 · confidence 5/52023-07-05

Summary

This paper proposes a new exploration method under maximum entropy RL settings. At each time step, the agent selects the action that maximises the expected entropy of the finite-length trajectory. The trajectory entropy is decomposed into two terms, based on variants of the predecessor representation and successor representation, respectively. The authors proposed separate training frameworks under discrete and continuous action spaces. The resulting agent is evaluated in grid worlds with different configurations.

Strengths

- The proposed decomposition of trajectory entropy objective is novel; - The authors provide comprehensive description of Q-learning and policy gradient training under discrete and continuous action spaces, respectively. - Empirical evaluations are comprehensive and coheres with the arguments in the paper;

Weaknesses

- The predecessor representation part (for the computation of the entropy of past trajectory) seems unnecessary and does not contribute to action selection, could the authors elaborate on this point? - The evaluations in continuous control tasks is still within the grid world environment, could the authors evaluate the proposed agent in standard exploration-demanding continuous control tasks;

Questions

See questions in Weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Yes.

Reviewer RPnY6/10 · confidence 4/52023-07-07

Summary

This paper proposes a novel exploration algorithm in RL by combining the successor representation with the predecessor representation and maximising episode-level entropy of state visitation. The proposed approach demonstrates improvement over the MaxEnt baseline on simple Gridworld and continuous control environments.

Strengths

- This paper presents a nice way to bridge the successor and predecessor representation to maximise entropy in the visitation of states in an episode. - The paper is clearly written and well motivated.

Weaknesses

- In the experiments, the proposed approach is only compared with the MaxEnt baseline. While this is a good baseline to compare with since the objective is similar, the standard exploration baselines are missing: 1) epsilon-greedy exploration, 2) count-based intrinsic motivation approaches like UCB, 3) random action baseline, and optionally 4) auxiliary objectives for exploration, like curiosity-based learning. While I agree with the related work section that these works learn Markovian policies while the proposed work learns a policy conditioned on the full history, a couple of them should still have been included, to place this algorithm in the overall landscape of exploration algorithms. - Since the successor representation and the predecessor representation in this case both depend on the full history, it is not possible to disentangle the impact they have on exploration and the evaluation metrics as there is no baseline included in which the successor representation depends only on the current state. - A relevant paper that is missing from the related work section, and also from the baselines, is "A First-Occupancy Representation for Reinforcement Learning" by Moskovitz et al, which is another state representation that indicates the time of first access of a state by the agent and has been shown to be conducive to exploration.

Questions

Please refer to the comments listed in the Weaknesses section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The authors have discussed general high-level ethical concerns with this line of work, but the limitations of the proposed approach have not been discussed.

Authorsrebuttal2023-08-12

We thank the reviewers for their time and feedback on improving our work. We are delighted that the reviewers find the decomposition of estimating state visitation distribution using successor and predecessor representations and proposed learning frameworks in Q-learning and policy gradient setting novel. We have attempted to address all concerns and significantly improved the manuscript as a result. Based on the feedback, we have 1) added baselines on continuous control tasks, 2) presented an experiment to show that the method can be scaled, and 3) shown an application where $\eta\psi$-Learning combined with existing methods can learn efficiently in sparse reward environment. We would be happy to answer any questions during the discussion period and hope the reviewers would be willing to increase the score. Thank you for your time, The Authors

Authorsrebuttal2023-08-14

We thank the reviewer again for their review. We have added more baselines, an experiment where the SR is only conditioned on the current state, and also experimented in a sparse reward environment to compare with other exploration-based methods. As the end of the discussion period is approaching, we kindly ask the reviewer to engage in further discussion and we hope the reviewer could adjust their score accordingly if all raised concerns are addressed. Thank you for your time,\ The Authors

Reviewer RPnY2023-08-17

I thank the authors for addressing the points I raised in my review, and have increased my score.

Authorsrebuttal2023-08-14

We thank the reviewer again for their review. We have elaborated on the importance of predecessor representation and present how the proposed method can be scaled to tasks with more complex state spaces. We would be happy to answer any further questions. As the end of the discussion period is approaching, we would like to kindly ask you to review the changes and assess whether the concerns are addressed. If so, we hope that you would be willing to increase your score. Thank you for your time,\ The Authors

Authorsrebuttal2023-08-18

Dear Reviewer L5xE, We hope that you've had a chance to read our rebuttal. As the end of the discussion period is approaching, we would greatly appreciate a reply as to whether our response and clarifications have addressed the issues raised in your review. Thank you for your time,\ The Authors

Reviewer gBvc2023-08-14

Response to rebuttal

I thank the authors for comprehensively responding to my comments. I feel that the paper should be accepted, and I am raising my score to 8.

Authorsrebuttal2023-08-14

We thank the reviewer again for their review. We have added ways to scale the proposed method and added more baselines. We also discussed on comparison with FB representation and estimation of state entropy instead of state visitation and updated the paper with these points. As the end of the discussion period is approaching, we would like to kindly ask you to review the changes and assess whether the concerns are addressed. If so, we hope that you would be willing to increase your score. Thank you for your time,\ The Authors

Authorsrebuttal2023-08-18

Dear Reviewer RqRN, We hope that you've had a chance to read our rebuttal. As the end of the discussion period is approaching, we would greatly appreciate a reply as to whether our response and clarifications have addressed the issues raised in your review. Thank you for your time,\ The Authors

Reviewer RqRN2023-08-19

After response

I am very sorry for my late reply. I want to thank the authors for their detailed comments on the points I raised. I am happy to keep my original positive evaluation, and I will recommend accepting this paper in the private discussion.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC