Summary
The authors consider the problem of generating sets of diverse and interesting multi-player games. They instantiate this problem in the subspace of board-game like games using a recent domain-specific language called Ludii. They then use a quality-diversity algorithm (specifically, map-elites with a language-model-based mutation operator) as their generation algorithm. They name their overall approach GAVEL.
For the mutation operator, the authors fine-tune a 13b codellama model with a fill-in-middle objective to reconstruct structured portions of Ludii games in a train dataset of ~500 games. For the behavioral characteristic, the authors use a set of game features provided by the game engine on which they have performed PCA (fitted on the train set). For the fitness function, the authors use a set of 3 filters allowing to remove invalid, trivial, severely unbalanced games (among others) and then compute separate scores along several metrics of playablility derived from domain insight and expertise (e.g. amount of time a player has more than 1 option to play, proportion of games that don't end in a draw, etc). The map-elites archive is seeded with 14 games novel to the mutation model.
The authors report qualitative as well as quantitative evaluations of map-elites on their domain. They demonstrate that the method outperforms (by QD-score) a version where portions of code that have been successfully mutated beforehand are preferentially mutated (GAVEL-UCB). They show that GAVEL is able to create games in novel cells compared with the original Ludii set. They showcase some their created games, selected from the set of promising games by human experts.
Strengths
* The paper tackles automatic game generation, which is an interesting area of research where modern machine learning methods have much to contribute;
* The paper is excellently written and very clear (to a reader with knowledge of the area at least). Presentation, pacing and discussion are well executed; it was a pleasure to read. The illustrations are nice;
* The methodology is very sound, all steps are explained in detail and would be simple to reproduce.
* The application of evolution through large model (ELM) like methods to game generation seem novel;
* The cherry-picked examples of games created by the method look engaging, demonstrating the interest of the method;
* The use of concept vectors are a representative behavioral characteristic (BC) in this domain;
* The fitness function, while domain-specific, is precise and captures fine-grained notions of enjoyableness of games;
* Several seeds of all experiments are given, allowing to quantify variability;
* The authors are very upfront about the limitations of their methods and the claims being made are well supported with evidence.
Weaknesses
* I worry that the contribution of the paper is limited. While using Map-Elites + LLMs on Ludii is novel, both the fitness function and the BC space used are very specific to games (of the kind presented in the paper), and the paper presents no algorithmic improvement. I would have welcomed methods that work across domains (for instance ask an LLM to self-refine a fitness function and BC space?)
* While generally, applying a method to a novel domain is interesting in its own right, I worry about the lack of comparative studies in the experiments of the paper. It is good to study map-elites on this new domain but what are the lessons learned? Where are comparisons to other methods, could other generation algorithms (like the ones cited in the related work, or sampling from the model repeatedly without QD, or few-shot prompting a large model with a larger context window) function just as well? Right now this paper reads more like a technical report than a scientific paper, since I do not feel I learned how this complex algorithm works on this domain by careful ablations and comparisons.
* There could be more information on the study of dynamics of evolution of games in the paper. Why did the authors stop at 500 evolution steps? Was the discovery process saturating? If yes, why? Could more OOD games have been discovered if the process would have been run for longer? If more samples would have been used to seed the archive?
Questions
### Questions
* line 104 did you try not removing these games, does anything bad happen?
* would CVT-mapelites have worked in this case (to avoid training the PCA on train set games and collapsing variation that doesn't exist in the train set?)
* Do you have ideas on how to characterize game diversity in other spaces than the BC space? How can we validate that this space adequately capture what humans mean by diverse games?
* finetuning vs in-context learning: why have you fine-tuned a smaller model instead of asking a large, capable instruction model to perform the task (with examples?) Did you experiment with this? (cost is a valid reason to try using a smaller model, but I am curious on how larger models perform on the task).
* Open question on fitness: could the various playability measures be collapsed to a single metric such as learnability, or a computational measure of interestingness, do you have ideas on how this could work or whether it is feasible? (See for instance the interesting but empirically ungrounded work in https://arxiv.org/abs/0812.4360, somewhat related to a recent formulation of open-endeness: https://arxiv.org/abs/2406.04268). This is beyond the current scope of the paper, but I am curious to know if the authors have any thoughts as to how this relates to game domains.
* Why to you think the UCB variant works less well?
* The limitation section discusses biasing mutation, do you have any idea on how this could work in practice?
### Suggestions
* (minor) The middle figure of figure 2 could show function expansion;
* l121 should have a footnote specifying when the training was performed because these things change very fast. (eg codestral can do fill-in-the-middle too now, but was released after the neurips deadline);
### Related work
Here is some work related to the current approach:
* There is a recent trend for game generation with LLMs, mostly focused on generating geometrical information. The seminal work here is MarioGPT (https://arxiv.org/abs/2302.05981) but there have been several works in this vein since, like https://arxiv.org/abs/2403.12014, https://arxiv.org/abs/2302.05817 or https://arxiv.org/abs/2406.04663
* On LLM-augmented QD, rainbow-teaming is also an important paper https://arxiv.org/abs/2402.16822
Limitations
The authors acknowledge both a large set of limitations of their work as well as its broader impact. I would add that an important limitation of the paper is that the proposed implementation of LLM-augmented map-elites lacks generality: fitness, mutation operator and BC all are specific to the Ludii environment.