Comparative Analysis of DISCO-DANCE and LSD
Dear Reviewer gakY,
We appreciate your feedback regarding the need for comparisons with algorithms such as DADS, MUSIC, and LSD. Initially, as discussed in our response, we classified skill discovery methods into two broad categories:
1. Algorithms that utilize $r_{skill}$ with a mutual information objective while focusing on constructing an effective auxiliary exploration reward $r_{exploration}$.
2. Methods that seek to redefine $r_{skill}$ beyond mutual information.
In our earlier communication, we positioned DISCO-DANCE in the former category, believing that its closest counterparts were the algorithms targeting the exploration reward. This informed our initial decision to exclude DADS, MUSIC, and LSD from our comparisons, as they were primarily aligned with the latter category.
However, upon your feedback and deeper reflection, we recognize the value of broadening our comparative analysis. Even if DADS, MUSIC, and LSD primarily fall into the second category, these algorithms may overcome the inherent pessimism associated with the mutual information objective. In response, we have decided to extend our comparative analysis with LSD to ensure a comprehensive evaluation of skill discovery algorithms.
Among those, we selected LSD, the algorithm known as the top performer among the reviewer has suggested. Due to time constraints, we narrowed our focus to the Ant-$\Pi$ maze, the environment where the agent suffer from exhaustive exploration.
| $r_\text{skill}$ | $r_\text{exploration}$ |state coverage |
| -------- | -------- | -------- |
| DIAYN | None | 22.50±3.34 |
| DIAYN | DISCO-DANCE | 39.00±4.85 |
| LSD | None | 38.80±3.34 |
| LSD | DISCO-DANCE | 45.80±3.34 |
The above table illustrates the performance comparison with LSD. A noteworthy observation is that, unlike in the 2D-bottleneck-maze (Fig.c in attached pdf), both DISCO-DANCE and LSD exhibit comparable performance in the Ant-$\Pi$ maze. This potentially underscores LSD's capability to mitigate the inherent pessimism of mutual information objective.
While the performance of LSD is noteworthy, it doesn't diminish the significance of DISCO-DANCE. This is because $r_{skill}$, when decoupled from the mutual information objective, and $r_{exploration}$ as guided by DISCO-DANCE, can work in tandem, complement each other. We tested this by combining DISCO-DANCE with LSD, which led to significant performance improvements. Thus, in environments that are challenging to explore, DISCO-DANCE can serve as a key role in improving exploration.
In summary, we acknowledge that our initial experiments primarily utilized MI-based algorithms, specifically DIAYN. Yet, as shown in our additional experiments, DISCO-DANCE not only outperforms LSD in both the 2D-bottleneck-maze and Ant-$\Pi$ maze, but also exhibits augmented performance improvements when integrated with LSD. We are also in the process of conducting experiments with MUSIC to provide a more comprehensive evaluation across various environments. In our revised manuscript, we will incorporate these results, along with additional ablation studies where DISCO-DANCE is utilized on top of different skill discovery objectives.
We hope our response alleviates your concerns.
Best, Paper 12428 authors