Summary
This paper proposes a benchmark for "active reasoning" titled Conan, where, instead of passively answering questions from provided context, an agent must interactively explore its environment to discover information. The authors differentiate such a task from so-called "passive reasoning" tasks such as video-language understanding where the visual input is directly fed into the model. Here, a model itself is responsible for exploring and attaining input that will help a model answer a question.
Conan is implemented in a Minecraft-like gridworld where a "detective" agent must identify activities that a rule-based, goal-oriented "vandal" agent completed, by using traces of the vandals' behavior in the environment. E.g. if a vandal's goal was to make a wooden pickaxe, it may have cut some trees down, and the task for the model will be to answer why the vandal decided to make a wooden pickaxe.
Two alternative methods are adopted to approach this task: one which uses an explorer agent, trained with an exploration reward, to generate evidence that is fed into a standard VLM, and another, "Abduction from Deduction", where one directly learns to predict the goal of the vandal from an inferred state trajectory.
Overall this is an interesting paper, and I think the dataset and benchmark will be valuable for the community. The models tested for this environment are relatively simple, not really making full use of the "joint exploration and reasoning" abilities that Conan purportedly tests, however. Moreover I do have some outstanding questions, and some issues with the experiments (specifically a missing baseline) that prevent me from assigning a higher score. However I'm open to changing my score after the author response and discussion period.
Strengths
- To my knowledge, the "active reasoning" component of Conan is an important area in vision/lang/RL, and is certainly underexplored in current embodied QA settings (although it is probably implicitly present in instruction following benchmarks and work like SayCan).
- The dataset seems high quality and should be a useful contribution for the field.
- Fairly sound experimental evaluation, comparing a variety of RL explorer agents with a variety of vision-language models.
Weaknesses
- **Missing negative control baselines.** The experimental results should compare evidence gathered by the trained explorer agents to a weaker negative baseline, e.g. a random untrained explorer policy or no explorer input at all. This is needed to convincingly show that the explorer agents are actually learning to explore in a way that is more beneficial than chance, at least for the downstream QA task.
- In general I'm definitely more interested in to what extent performance is bottlenecked by good exploration on this task, rather than fixing the explorer evidence and trying subtly-differently-trained large VLMs.
- As mentioned by the in the conclusion (L376-377), one of the key promises of Conan is that strongest performance on this benchmark should intuitively be achieved by models that jointly learn to reason and explore, but right now a decoupled two-stage process is adopted—first training an exploration model with an oracle "find trace" reward, extracting relevant keyframes, then training a VLM on top of the frozen policy. As a result, models trained for this task (for now) look like typical embodied QA models doing passive reasoning (just with the extra step of training an explorer to get the vision input). Of course, it's not strictly necessary for a benchmark paper to include an strong novel model for the benchmark, but without it we aren't able to evaluate whether Conan performance numbers have **headroom** for more sophisticated approaches to joint exploration and reasoning. (It seems like this should be the case, but we can't really be sure).
- Some more outstanding questions (see next section)
Questions
- The idea of answering questions via "traces" left behind in the environment is a little hard to grasp and could use more explaining. There are some example traces in Figure 1b but these are small figures and aren't explained that much. The authors could spend some more time walking through example "traces" and more fully convince readers that the traces indeed provide answers to some of the more subtle questions in the benchmark (e.g. how do traces indicate that the vandals goal is to "get the diamond", couldn't the vandal have alternative goals like get iron/coal?).
- It might be valuable to have an "oracle" setting to demonstrate that with perfect trace evidence and enough training models can 100% this task—or else explain why such an oracle doesn't exist (e.g. what causes the lagging performance even for the "Ideal Explorer" setting in Table 3? Is it an amount of training data issue? Are there still limitations as to the quality of the trajectory attained even by the ideal explorer, and is there an even more suitable oracle for this?)
- Related to this, can authors further explain L308 ideal explorer: "visible to the ground-truth vandal’s trajectory"? What does "visible to" mean—shouldn't the ideal explorer literally *be* the ground-truth vandal's trajectory?
- More details could be provided on the specific kinds of generalization splits tested here. At test time, is there a subset of abductive reasoning tasks held out that the model has never seen? Or does the model see the same questions it has seen during training, just in unseen environments? Do the test environments differ systematically from those during train (or is it possible to induce such a )
- Evaluating (and supporting) such compositional/systematic generalization splits would greatly increase the appeal of the benchmark.
- (Related to the above) Do authors have an intuition as to whether test-time performance drop is more due to the VLM's inability to generalize, or the explorer's inability to generalize and generate good evidence for unseen enviroments (if test does indeed have unseen environments)? Is it possible to disentangle the two using different evaluation splits in the environment?
- In Figure 4, does a score of near 100 as attained by eg TRPO indicate that the explorer indeeds find all traces (+100) reward all of the time? If not, is there an explicit measure of that?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.