Adaptive Exploration for Data-Efficient General Value Function Evaluations

General Value Functions (GVFs) (Sutton et al., 2011) represent predictive knowledge in reinforcement learning. Each GVF computes the expected return for a given policy, based on a unique reward. Existing methods relying on fixed behavior policies or pre-collected data often face data efficiency issues when learning multiple GVFs in parallel using off-policy methods. To address this, we introduce GVFExplorer, which adaptively learns a single behavior policy that efficiently collects data for evaluating multiple GVFs in parallel. Our method optimizes the behavior policy by minimizing the total variance in return across GVFs, thereby reducing the required environmental interactions. We use an existing temporal-difference-style variance estimator to approximate the return variance. We prove that each behavior policy update decreases the overall mean squared error in GVF predictions. We empirically show our method's performance in tabular and nonlinear function approximation settings, including Mujoco environments, with stationary and non-stationary reward signals, optimizing data usage and reducing prediction errors across multiple GVFs.

Paper

Similar papers

Peer review

Reviewer 3wne6/10 · confidence 4/52024-07-10

Summary

This paper presents a novel method named GVFExplorer for efficiently evaluating multiple Value Functions with different policy (GVFs) in parallel using off-policy methods. It adaptively learns a single behavior policy that minimizes the total variance in return across GVFs, thus reducing required environmental interactions. The method uses a temporal-difference-style variance estimator and proves that each behavior policy update decreases the overall mean squared error in GVF predictions. The performance of GVFExplorer is empirically validated in various settings, including tabular and non-linear function approximation, with stationary and non-stationary reward signals.

Strengths

- idea of choosing behavior policy for simultaeneous learning of multiple policies is quite novel, and seem to be able to be applied to some applications. - systematic way of deriving algorithm is novel and interesting, and makes sense as well

Weaknesses

- More experiments would help understand the behavior of the algorithm. - uniform policy being the best baseline does not seem to be a good baseline choice. Having some more competetive baselines, e.g. ablations, would have been much better. - experimented environments seem too synthetic. would like to see results on typical (and challenging) RL environments, e.g., mujoco.

Questions

- The paper starts with minimizing MSE, and argues that it uses unbiased IS estimation, leading to a policy that minimizes the variance of return. However, GVF in algorithm is estimated with $Q_\theta$, which has been learned with expected Sarsa, which is a baised estimate given that the target contains $Q_\theta$ instead of GT $Q^\pi$. How do we ensure that the analysis above also works in algorithm 1? What happens if our function approximator is crude so that the bias is large? - Why does PER drastically improve the performance of GVFExplorer? If we have infinite number of data in our experience replay, GVFExplorer would have very similar effect as PER. In my opinion, it seems more natural to have reduced effect of PER as we are sampling hard states more with GVFExplorer, and it is what PER tries to do as well. (it also does not go well with the rest of the paper, as PER is about training efficiency where GVFExplorer is about the choice of behavior policy)

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors adequately addressed the limitations.

Reviewer A7c77/10 · confidence 4/52024-07-11

Summary

This paper propose a new algorithm to solve the general value function evaluations problem. In essence, GVFs can be seen as high dimensional value functions. The authors propose a temporal difference learning algorithm that minimizes the overall variance in the return distribution, in the hope to improve the behavior policy for better exploration, such that the samples the behavior policy produces better suffices for off-policy evaluations for each GVFs.

Strengths

1. The idea of minimizing the variance of the return of the behavior policy is novel. While the idea might now be groundbreaking, it is new in this field. 2. Incorporating a temporal difference to approximate the overall variance of the return of the each general values are interesting. Indeed, for large scale problems, TD is a better solution overall. 3. The derivation and analysis of the algorithm is sounds and rigorous. 4. Overall the paper is clearly presented.

Weaknesses

1. The core idea behind this paper is to propose a solution to solve the data collection problem. This paper does answer the question of how to minimize the variance of the returns, but fail to convince me entirely why we should do that at the first place, either through proofs, or empirical investigation. I think this is the biggest weakness of the paper. 2. The experiment section is nice and clear but the problem class is a bit simple (gridworld). For a paper without strong theoretical results, experiments are usually expected to have more materials. In this sense, the results are not very convincing.

Questions

Please see weakness above.

Rating

7

Confidence

4

Soundness

4

Presentation

3

Contribution

3

Limitations

Yes.

Reviewer pomH6/10 · confidence 4/52024-08-08

Summary

This paper presents a new algorithm for collecting data needed to learn multiple GVFs in parallel. By focusing data collection on high-variance (s,a) pairs, an agent is able to collect data that will reduce the variance of estimated GVFs. The authors contribute a sort of contraction-mapping proof that using their algorithm will result in non-increasing variances.

Strengths

The algorithm is simple, with some reasonable theoretical properties. The paper addresses a good problem. The paper is very well written The experiments seem reasonable and informative The method performs well against other baselines

Weaknesses

I have one minor-but-important quibble about the paper. In Thm. 4.2, the authors prove that aggregated variance is "<=" upon successive iterations. However, in the english description of the result, the authors state that the aggregated variances "decrease with each update step". This is NOT what you proved - you proved that aggregated variance "does not increase." The same claim is made in the abstract, and again in the conclusion. I think it's important to be clear on this point, so I would ask the authors to rephrase this.

Questions

I wonder if there are potential degeneracies in the algorithm. For example, if a behavior policy never tries a certain action, it seems like the M's for that action will be 0, and the new behavior policy will assign 0 probability to taking that action in the future, leading to a situation where you never get the data you need. Similarly, if a certain state is never visited (in the tabular case), would the estimate of M be zero? If so, does that imply that there is an unstated constraint on the initial behavior policy -- something about exploring states and actions sufficiently? A related question: if the cumulant function is sparse, is it possible to not get enough non-zero data to get non-zero M estimates? (by the way, it's these sorts of questions where I can kind of see that degeneracies may arise that will never get ironed by your algorithm, which is why the difference between "<" and "<=" in your proof is important)

Rating

6

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

It seems like a central limitation of the work is a disconnect between what was theoretically proven (which relies on a perfect knowledge of the variances M), and what will happen in practice (the M's must be estimated). It would be nice if the paper outlined what happens to the algorithm as a function of the error in the estimates of M.

Authorsrebuttal2024-08-09

Author's Response to Reviewer pomH

Thank you reviewer for your thoughtful and constructive feedback. We provide a detailed response to the asked questions below. 1. **“Clarification regarding Theorem 4.2”** We appreciate your careful attention, you are correct that Theorem 4.2 demonstrates that the aggregated variance "does not increase" with each update, rather than strictly "decreases." due to "<=" sign in the proof. We will make the necessary revisions to the paper to accurately reflect this. 2. **“Regarding question on potential degeneracies and exploration of algorithm”** Thank you for raising this important point. Just like in standard RL, to obtain reliable estimates—whether for Q-values or variances—it's important to have at least initial exploration of different areas of the state-action space. This can be achieved by incorporating any standard initial exploration technique. In our work, the proposed algorithm assumes sufficient initial exploration to gather necessary data. We address this by initializing M values to a non-zero constant across all state-action pairs and also allowing epsilon exploration, where epsilon decays over time. This ensures agents visit a wide range of state-action pairs early on, preventing issues of zero variance for unvisited state-action pairs. We added epsilon-exploration for all algorithms including baselines. 3. **“Effects on M with sparse cumulant”** In scenarios with sparse cumulants, the non-zero initialization of M ensures that all states are visited, providing a fair opportunity to correct M estimates over time. 4. **“What happens to the algorithm as a function of the error in the estimates of M?”** Similar to any TD-based algorithm, the empirical version of our approach relies on initial estimates, which will be imperfect. If the initial M estimates are incorrect, the TD error will indicate this, either positively or negatively. The behavior policy update will then be influenced by these M estimates, leading the agent to gather new samples as it interacts with the environment. As agent collects more data, the M estimates will improve—either increasing or decreasing for specific states—allowing the behavior policy to adjust and correct its sampling strategy accordingly. If an M estimate is very small compared to its true value, the agent will first focus on states with higher M estimates, correct those values, and then revisit the poorly estimated states to update their M estimates. This iterative process is analogous to TD-based Q-learning updates in standard RL. Additionally, if the target policies have non-zero probabilities, our behavior policy incorporates a small epsilon probability over those state-action pairs. This approach is also supported by Lemma 6.1 which requires a bounded difference between the behavior and target policies to ensure that the variance function remains well defined. For Mujoco experiments, we added KL term to limit this divergence. We will include these explanations in the camera-ready version to provide a practical understanding of our algorithm.

Reviewer A7c72024-08-11

Thank you for the response

I thank the authors for the response. Judging from other reviews and the added experiments, I will keep the same score for now.

Reviewer 3wne2024-08-12

Thanks for the response. I raised my score.

Authorsrebuttal2024-08-13

Author response to Reviewer 3wne

Hello Reviewer, thanks for responding and raising the score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC