Response 2/2
# Justification for SFL & Different Definitions of Learnability
We believe the intuition for learnability is simple, easy to understand, and our score function works well empirically compared to prior approaches. However, we are not claiming that learnability is the be-all and end-all for score functions; we fully agree that there may be other variations of it that could perform better, and we are glad the reviewer raised this important point for future work.
Furthermore, we would like to point out that the regret approximations used in current UED methods (e.g., MaxMC and PVL) are not particularly principled and are at best a loose approximation to regret. Indeed, in [1], six possible functions are trialled with their final choice based solely on empirical results. Prior work has also not investigated how closely these metrics correlate to true regret, and recent work has not investigated any alternatives to PVL and MaxMC.
Further justification for our score function is presented below in three parts:
**First, we wish to emphasise that our learnability definition is deeply connected to a large body of prior work in learning theory and curriculum development.** Intuitively, many curriculum methods (including SFL) aim to obtain experiences within the agent's "zone of proximal development", i.e., those that are neither too easy nor too hard [2].
Works such as [3] use hardcoded reward thresholds to define a goal that is of appropriate difficulty. Similarly, POET [4] also uses predefined reward thresholds to filter environments. SFL, meanwhile, directly uses the success rate of the agent on the level to compute the score.
**Second, we believe our empirical results are a strong justification of SFL**, with the reviewer agreeing that "SFL outperforms UED". We would also note that most prior peer-reviewed work in UED uses two ([5, 6]) or three ([7]) environments, and we have strong empirical results in four.
**Third, we have run several other variations on learnability, all of these assign a level a score of zero when $p=0$ or $p=1$** (in JaxNav):
1. Uniform sampling of levels with $0 < p < 1$.
2. Learnability linearly increasing from $p=1$ to $p=0$.
3. Learnability linearly increasing from $p=0$ to $p=1$.
We find that variation 2. (which is an approximation of true regret) performs similarly to the default definition of learnability.
Variation 1. performs worse than our approach, but still outperforms all UED methods (albeit by a smaller margin than SFL).
Finally, variation 3. performs significantly worse, and we believe this is due to this score function prioritising easy levels, which provide less opportunity for the agent to improve. We will include these results in our updated Appendix.
While we focused on $p(1-p)$, these results (and the previous results with different learnability peaks) show that our method performs robustly under other reasonable learnability estimates, demonstrating the potential of learnability in the UED setting and the robustness of SFL.
While we agree that the exact choice of learnability metric is still an open question, we believe that all our claims are justified empirically, and our results make an impactful contribution to the UED community.
**References**
[1] Jiang, Minqi, et al. "Prioritized level replay." International Conference on Machine Learning. PMLR, 2021.
[2] L. Vygotsky. Interaction between learning and development. Readings on the Development of Children, pages 34–40, 1978.
[3] Carlos Florensa, David Held, Xinyang Geng, Pieter Abbeel. ICML, 2018
[4] Wang, Rui, et al. "Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions." arXiv preprint arXiv:1901.01753 (2019).
[5] Dennis, Michael, et al. "Emergent complexity and zero-shot transfer via unsupervised environment design." Advances in neural information processing systems 33 (2020): 13049-13061.
[6] Jiang, Minqi, et al. "Replay-guided adversarial environment design." Advances in Neural Information Processing Systems 34 (2021): 1884-1897.
[7] Parker-Holder, Jack, et al. "Evolving curricula with regret-based environment design." International Conference on Machine Learning. PMLR, 2022.
# Conclusion
Once again, we appreciate the reviewer's engagement in the review process, and their helpful suggestions. We believe these results further strengthen our paper.