Operating in the real-world often requires agents to learn about a complex\nenvironment and apply this understanding to achieve a breadth of goals. This\nproblem, known as goal-conditioned reinforcement learning (GCRL), becomes\nespecially challenging for long-horizon goals. Current methods have tackled\nthis problem by augmenting goal-conditioned policies with graph-based planning\nalgorithms. However, they struggle to scale to large, high-dimensional state\nspaces and assume access to exploration mechanisms for efficiently collecting\ntraining data. In this work, we introduce Successor Feature Landmarks (SFL), a\nframework for exploring large, high-dimensional environments so as to obtain a\npolicy that is proficient for any goal. SFL leverages the ability of successor\nfeatures (SF) to capture transition dynamics, using it to drive exploration by\nestimating state-novelty and to enable high-level planning by abstracting the\nstate-space as a non-parametric landmark-based graph. We further exploit SF to\ndirectly compute a goal-conditioned policy for inter-landmark traversal, which\nwe use to execute plans to "frontier" landmarks at the edge of the explored\nstate space. We show in our experiments on MiniGrid and ViZDoom that SFL\nenables efficient exploration of large, high-dimensional state spaces and\noutperforms state-of-the-art baselines on long-horizon GCRL tasks.\n