Emergent Communication for Rules Reasoning

Summary

This work investigates the emergent communication framework for reasoning rules. That is, unlike prior studies that focus on communication about perceived low-level contexts, this paper proposes a cognition-oriented environment to encourage agents to reason and communicate about high level-rules. To this end, it introduces a new interesting and unbiased benchmark, rule-RAVEN. This benchmark, as opposed to the original one (I-RAVEN) avoids overfitting and pushes the agents to have an actual communication protocol. The authors show, with different experiments, that agents are able to succeed in the reasoning tasks and develop a compositional and semantically stable language.

Strengths

The authors introduce a well-thought benchmark that could be beneficial for future works to analyze the content of emergent languages. This benchmark, which is a modification of I-RAVEN, forces agents to develop an actual communication protocol. They perform the needed ablation to show its benefit compared to I-RAVEN. Furthermore, this paper is well-written, and a detailed description of the setting, and hyper-parameters are provided (on top of the code).

Weaknesses

The main weakness of this work is its motivation. As stated in the paper, the goal of the emergent communication framework is to: - either study the origin of the human languages and/or - develop intelligent communicating artificial agents It is unclear what this work's position is. If the former, is there a theory that our language emerged to communicate about a high-level reasoning task? If so, can this line of work be clarified in the paper? If the goal is to develop communicating agents, communicating about visual inputs is more practical for human-agent interactions.

Questions

Can you explain further the experiments of the paragraph "Rule-RAVEN dataset" (line 260)? In particular, I don't understand why we have an "unsuccessful" communication game with "rule-RAVEN" dataset if the speaker was already trained (in a two-stage setting). That is, in this training regime, the speaker's language is not random, and the listener should be able to succeed in the communication game (or at least have a good enough accuracy) without modifying the speaker's language). Maybe you can elaborate more on the stage 1 training? Also, you state that "for the listener’s train accuracy still achieves ~0.9 even if the speaker’s message is completely ignored". How do you check if the speaker's message is completely ignored?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

.

Summary

This paper takes the ever-popular Lewis signalling game for emergent communication and studies experiments on rule-focused communication as opposed to the perception-focused communication of prior work. In particular, it uses a modified version of Raven's progressive matrices to formulate a signalling game directly on attribute-value vectors which requires pattern recognition/reasoning to complete. The authors find that agents can learn to communicate when using a two-stage curriculum which essentially pretrains the speaker for more stable communication at the beginning of the communication stage. The experiments demonstrate that the resulting emergent language correlates better with the underlying rules of observations rather than the individual observations themselves.

Strengths

## Originality - `[major]` Addresses the signalling game from a new perspective, i.e., reasoning instead of perception. - `[minor]` Introduces a new dataset. ## Quality - `[major]` Presents good variety of empirical evaluation with clear results - `[minor]` Presents different levels and senses of "generalization". ## Clarity - `[minor]` Details for implementation are presented without being overwhelming ## Significance *See Originality.*

Weaknesses

The paper is relatively complete, but what keeps my rating from being higher is that there is relatively sparse comparison with prior work. Such comparison would better contextualize the results and increase its significance. For further details, see the *Questions* section of the review.

Questions

- How does this work compare with prior art which also uses categorical variables for a signalling game, even if there is no reasoning element to the game? - What are the particular effects on an emergent language of a reasoning-focused game versus one that is perception-focused? What sort of inductive biases are present in the task that has a downstream effect on the language? ## Minor Comments/Questions - `Line 46` "inner": typo? - `Line 46, 86` "inter": typo? - `Line 122` Do not use curly braces for ordered sequences; use parentheses. - `Paragaph @ 119` Why is it necessary for there to be no ambiguity? This sort of ambiguity shows up frequently in human communication. - `Line 251` "cause" -> "because" - `Paragaph @ 246` I would use "shows" or "demonstrates" instead of "proves" since it is not a formal, mathematical proof. - Use the default LaTeX placement of tables/figures instead of `[h]`; the former is less distracting.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

N/A.

Summary

This paper proposed a new environment along with the training framework for emergent communication of abstract rules. They designed a context generation pipeline rule-RAVEN to avoid overfitting and a two-stage curriculum training method for more stable convergence. They evaluated the emerged language from the perspectives of generalization and transfer learning.

Strengths

1. This paper proposed a new research angle of abstract rule reasoning for emergent communication. The context requires the agent to go beyond the low-level perceptual features and communicate more abstract rules. 2. The candidate pool is smartly designed to motivate agents to extract rules from the context. 3. A suite of comprehensive evaluations is designed to measure the generalization of the emerged languages.

Weaknesses

1. Structural requirements of the new benchmark may need to be further explained: I am not clear about why the rules must be unambiguous. From my understanding, though the multiple rules can be applied to the current context, as long as the agents can communicate either of the rules, the receiver should capture the correct candidate? Though Figure 4 demonstrates the receiver can select the candidate correctly without the sender’s message trained with rule-RAVEN, further experiments/explanations are still needed to show that: a. The communication training benefits from the structural or functional requirement or both. b. The language that emerged using the I-RAVEN dataset is not/less generalizable/compositional/transferable. 2. Sender’s rule reasoning and perception encoding are entangled. In the first stage, both $g^S$ and $f^S$ are trained. Though the training data is not shown in the communication stage, it will still introduce structural information because of the term $\mathbb{1}(r_i, m_i)$. a. does that require the length of the messages to equal the size of the rules?

Questions

In Sec 5.1, how distance(rule) and distance(panel) are computed? Just want to clarify whether these two distances are comparable.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

1. Though the goal of the task is to emerge the language for abstract rules, it will also be interesting to know whether the receiver can learn to induce rules after the communication (instead of applying rules during the communication, not required experiments). Similar to ETL, you can test the accuracy of the reasoning problem on the communicated receiver without further training. 2. As the author mentioned, the current context input is a structured symbol. It will strongly encourage compositional language. It will be interesting to know how agents can emerge languages in a raw pixel input. 3. Can the emerged languages generalize to contexts with different attributes?

Summary

This paper introduces a novel setting for abstract reasoning (i.e., RAVEN problems) by proposing a speaker-listener framework for communicating higher-level abstract rules. The authors propose an unbiased dataset (rule-RAVEN) to overcome overfitting in the original RAVEN-family datasets (I-RAVEN), and propose a two-stage curriculum agent training method for successful communication. Experiments have shown the efficacy of the curriculum training for solving the rule-RAVEN and out-of-distribution generalization.

Strengths

- I like the idea of both: i). introducing communicative game settings to abstract reasoning tasks, and ii). see how higher-level relational abstractions (instead of low-level perceptual features) emerge in communicative games. The limited capacity communication channel formulation can lead to emergent abstractions for problem-solving, including more powerful representations for abstract reasoning and concept learning. Previous attempts in drawing are good cases but not complex enough to depict the importance of abstraction and emergent language. This preliminary trial on RAVEN tests sets a suitable problem formulation for emergent communication in abstract reasoning. - The rule-RAVEN dataset effectively mitigates the existing bias in the I-RAVEN dataset, making the speaker-listener communication valid. - The paper is well-written and easy to read. The flow of writing in section 5 is also appropriate for addressing potential concerns for readers.

Weaknesses

Although I like the task settings in this paper, the experiment and proposed methods appear to have some weaknesses. I list them as follows: - The communicative formulation is very similar to Mu & Goodman, 2021. It seems this work (communicative RAVEN) is a special case of generalization, shifting from learning object-centric, attribute-level concepts (e.g., shape red or blue) to learning relational concepts (number-increasing). Authors should address more comparisons to these existing formulations. - The evaluation for emergent language is still quite limited. For example, can you probe the learned language to see if it can be linearly projected to some algebraic representations for relational concepts (e.g., the "number increasing" concept can be described as a multiplication matrix in Zhang et al., 2022) or just explicitly manipulate them and see if they have some language-like syntax or compositionality emerged. - The use of symbolic RAVEN and two-stage curriculum training (with the first stage supervised learned) made me doubt the applicability of this communicative method to more complex or real-world tasks. For example, Mu & Goodman, 2021 used a real-world dataset, pixel input, and end-to-end training. refs: 1. Mu, J., & Goodman, N. (2021). Emergent communication of generalizations. Advances in Neural Information Processing Systems, 34, 17994-18007. 2. Zhang, C., Xie, S., Jia, B., Wu, Y. N., Zhu, S. C., & Zhu, Y. (2022, October). Learning algebraic representation for systematic generalization in abstract reasoning. In European Conference on Computer Vision (pp. 692-709). Cham: Springer Nature Switzerland.

Questions

See the weakness section. I am open to changing my score, so I hope the authors can address these concerns.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors did not address the limitations.

Summary

This paper proposes an emergent communication game over abstract visual concepts, inspired by Raven's progressive matrices tests. The basic idea is to evaluate neural speakers and listeners on a communication game, where the speaker sees a collection of images encoding some abstract rule (e.g. "number of objects in the image is increasing"); the speaker must then generate a message that allows a listener to complete an unseen sequence. The authors show that agents trained to play this game indeed seem to learn to communicate the abstract rules for which they are trained for, as measured by intrinsic measures of language compositionality and ease of transfer to harder tasks.

Strengths

- This is an interesting dataset and interesting problem in emergent communication which may be useful to the community. It indeed explores more abstract visual concepts than in existing work (though note that novelty over the existing EC literature is overclaimed; see Weaknesses). - Careful controls for dataset difficulty (ensuring one distinct feature that can be used to solve each task; ensuring "hard negative" rules) show the authors' care to making sure this is a well-constructed dataset, including an analysis of to what extent existing - Interesting experimental analysis shows that models seem to be (to some extent) communicating abstract rules, rather than superficial input features.

Weaknesses

- Only a synthetic dataset consisting of clean symbolic inputs is evaluated. One could imagine more realistic settings requiring communication of rules at least over synthetic visual inputs, if not more realistic visual concepts. Similarly, there is no exploration of downstream transfer to other tasks that perhaps don't involve emergent communication, e.g. instruction following or visual reasoning. While this does not preclude publication, there's not a lot one can gain from this paper as it relates to actual realistic ML tasks. If the paper were to be rejected, IMO it would likely be because the experiments are just a little too synthetic/marginal to be useful to the broader NeurIPS community. - The claim that existing work in EC does not at all care about expresing abstract generalizations or rules is a bit overblown. Separating inputs given to the student and teacher, so as to facilitate communication of abstract concepts, was introduced as early as Lazaridou (2017), recurs in Choi et al., Kiela et al., etc. Mu and Goodman (2022) also propose generalizations over abstract visual concepts involving multiple visual inputs, which is very similar to the task presented here. I do think the present work makes some interesting contributions over the existing literature, in that it is even more abstract, but the relation to existing work needs to be made more clear. Many of these papers are not discussed in detail and simply bucketed as "forcing agents to descrie low-level features of images" (L31-32) which I believe is false. Section 2 Emergent Communication also completely neglects to discuss such efforts in the EC community. - I think it's important for footnote 1 to be made more clear in the text, i.e. that this is not a grounded communication game over real images, despite many of the introductory figures seemingly suggesting this. ## Minor - Title of paper and title on OpenReview do not match - spaces between text and citations would be ideal - It'd be interesting to see how pretraning agents on such visual reasoning communication tasks might improve performance on downstream visual reasoning tasks such as ARC (Chollet et al., ?) - The description of the paragraphs in L119 and L128 as "structural" and "functional" requirements is a little confusing and nonstandard to me—it's not clear what structural and functional mean here. It might be appropriate for example to refer to the "functional requirement" as sampling "hard negative distractors", as is used in the terminology for contrastive learning for example. In other words, distractors should be sampled carefully so as to represent close but not quite correct rules that force the speaker and listener to communicate precisely the right rules. - L171 "directly from the sketch" -> "directly from scratch"?

Questions

- Did authors try varying the number of context panels given to the speaker? Or even show the partial sequence given to the listener? Wonder how this affects the languages' propensity to communicate abstractions; e.g. if the speaker sees the listener's partial sequence, does the language still communicate the abstract rule, or does the speaker internally learn the abstract rule but nevertheless convey the perceptual features (e.g. "single triangle")?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

2 fair

Limitations

yes

Thanks

Thanks to authors for their detailed response to my review, and for the follow up experiments which are quite interesting. Although I still think the task and domain are synthetic, I appreciate the inclusion of a more interesting downstream transfer task, and the author's rebuttal has solidified the difference between this work and related work. I'll increase my score to a 6.

Thanks

Thank the authors for the detailed response. I decide to increase my rating by one. I recommend acceptance of this paper.