Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments

Response to Reviewer TgUR (Comment 1-2)

Thank you very much for your detailed and thoughtful reviews. We have extensively revised our paper in light of the feedback received. These revisions aim to clarify the paper's positioning and enhance the readability of our experimental results. ## Reviewer's Comment 1 > The authors do a good job noting related prior literature, but I remain somewhat confused by the position of this paper relative to such literature. ... Thank you for point it out. We added the positioning of our paper, compared to the related studies Resnick et al. (2020); Chaabouni et al. (2021); Tucker et al. (2022). Perhaps another problem was the beginning of Sec 3: `The main proposal of this paper is redefine the objective of signaling game as (beta-)VAE's`. We modified the beginning of Sec 3 too. Moreover, we made the table to clarify our positioning, which is put to Appendix F in the revised version. ## Revision for Comment 1 At Paragraph "EC as Representation Learning" in Sec 6 "Related Work", we added: > (***Revision***) Resnick et al. (2020) formulated the objective as ELBO. However, they defined messages as fixed-length binaries and agents as non-autoregressive models, which is not applicable to our variable-length setting. Moreover, they do not discuss the prior distribution choice; they seem to have adopted Bernoulli purely for computational convenience. In contrast, we indicated that their choice influences the structure of emergent languages. At Pragraph "VIB" in Sec 6 "Related Work", we added: > (***Revision***) Note that messages are defined as single symbols in their settings. In contrast, messages are of variable length in our setting to discuss the structure of emergent languages such as ZLA and HAS. In Sec 3, we modified the first few sentences: > (***Revision***) ~~The main proposal of this paper is to redefine the objective of signaling games as (beta-)VAE's, i.e., the evidence lower bound (ELBO) with an additional KL weighting hyperparameter $\beta$.~~ > In this section, we first redefine the objective of signaling games as (beta-)VAE's, i.e., the evidence lower bound (ELBO) with a KL weighting hyperparameter $\beta$. Next, we show several reasons supporting our ELBO-based formulation. In Appendix F, we added the table to clarify our positioning. The table shows the characteristics of the studies in terms of: - whether their formulation is based on a generative perspective such VAE and VIB, - whether message length in their settings is variable, - whether they study the compositionality of emergent languages, - whether they investigate if emergent languages follow ZLA, - whether they investigate if emergent languages follow HAS, and - whether they are aware of the existence of prior distributions and carefully choose an appropriate one. Each related study stasfies some of them, while it does not satisfy the others. In contrast, our paper satisfies all of them. ## Reviewer's Comment 2 > I found the results somewhat difficult to read. Figure 2 contains the main results, and with enough flipping between pages, I could eventually figure out how to interpret them, but generally I encourage authors to make figures moreself-contained. For example, listing a baseline as BL1 is not as informative as using a name/label that actually describescharacteristics of the baseline (e.g., conventional + entropy). > ... > ... > Appendix D would greatly benefit from a little bit more text explaining what the graphs present. There are also some sentences that need editing (e.g., "threshold is set to 0.25.") Thank you for this feedback. ## Revision for Comment 2 To make Figure 2; 3; 4 more readable and self-contained, we added more information to the legend and included the explanation on the evaluation metrics and the baseline methods in the captions. As a result, the caption (in Figure 2) became as follows: > (***Revision***) Results for $n_{\textrm{bou}}$ (C1), $n_{\textrm{seg}}$ (C2), $\Delta_{w,c}$ (C3), C-TopSim (C3), and W-TopSim (C3) are shown in order from the left. The x-axis represents $(n_{\textrm{att}},n_{\textrm{val}})$ while the y-axis represents the values of each metric. The shaded regions and error bars represent the standard error of mean. The threshold parameter is set to $0$. The blue plots represent the results for our ELBO-based objective $\mathop{\mathcal{J}}\nolimits_{\textrm{ours}}$, the orange ones for (BL1) the conventional objective $\mathop{\mathcal{J}}\nolimits_{\textrm{conv}}$ plus the entropy regularizer, and the grey ones for (BL2) the ELBO-based objective whose prior is $P_{\alpha}^{\textrm{prior}}$. > ...... (*For the rest part of the revised caption, see our response to Comment 6*)

Response to Reviewer E7HT (Comment 1-4)

Thank you for your thorough and insightful review. In response to the comments provided, we have diligently revised our paper. Your constructive feedback has been instrumental in enhancing the readability and academic value of our work. ## Reviewer's Comment 1 > (major) A critical part of the paper is the "prior" within a VAE or signalling game, but I did not get a concrete sense of what this prior actually is in the context of a signalling game with neural network-based agents (I expand on this in "Questions"). As a result, it makes me unsure how well the theoretical claims actually apply to a real setup. We greatly appreciate your feedback regarding the prior in the context of EC signalling game. (Please see our response to Comment 3; 9; 10; 11; 12) ## Reviewer's Comment 2 > (major) The paper, I think, tried to do too much, and ends up not spending enough time on the core claims, namely, the signalling game can be re-analyzed as a VAE. I think the paper would benefit greatly from cutting away all but the essential claims and going through those more slowly and thoroughly. We value your suggestion to focus more on the core claims of our paper. In the revised version, we have taken the reviewers' comments into consideration and made efforts to improve the quality of the paper. We have removed unnecessary text and moved some parts to the appendix to focus more on the essential aspects. We hope you will find the revised version more satisfactory. Roughly speaking, our core claims are made of two components, which are (somewhat philosophically) of mutual dependence. Our core claims are: 1. The conventional EC signaling game is analogous to (beta-)VAE. 2. By explicitly re-defining EC signaling game as (beta-)VAE, the structure of emergent languages improves. Thanks to the 2nd claim, - the 1st claim becomes more than a mere theoretical result. Thanks to the 1st claim, - the 2nd claim becomes more than an arbitrary "hack" of the evaluation criteria (The arbitrary hack might be a questionable practice as a simulation of language emergence, in a scientific sense). For a more detailed decomposition of our claims, please refer to our response to Comment 4. ## Reviewer's Comment 3 > What exactly is the "prior" in the emergent language game? I understand that it is implicit, but does that mean that is embedded in the objective function (i.e., the term is constant)? Or is it instead the case that the sender's architectural biases represent the prior? We appreciate your question regarding what exactly the prior is. This is an important point to clarify our storytelling. In terms of the conventional EC signaling game, yes, the prior is constant. Specifically, it is a uniform distribution over a message space. (In contrast, we regard it as a language model) (See also our response to Comment 9; 10; 11; 12). ## Reviewer's Comment 4 > In addition to the theoretical analysis, what else can the authors point to to support the claim that an EC signalling game is analogous to a VAE? Thank you for questioning that point. Our contribution is indeed more than the theoretical analysis of the game, including e.g., - a new idea for EC setting (i.e., prior as a language model), - empirical results (for ZLA and HAS), and - relationship to the linguistic concept (i.e., surprisal theory). Our overall claims are as follows: First, as you mentioned, - ***As an interesting math problem***, EC signaling game is similar to (beta-)VAE (Sec 3.1; 3.2). Importantly, - By interpreting EC signaling game as (beta-)VAE, ***we can reveal the existence of implicit prior*** (Sec 3.1). Furthermore, by having revealed the existence of implicit prior, - We can appropriately re-define the prior distribution to have a ***good influence on the structure of emergent languages***, namely ZLA/HAS (Sec 3.3; 3.4; Sec 4). - We can naturally introduce a ***language model (=prior distribution) explicitly in the simulation of language emergence*** (Sec 3.4). Moreover, by regarding the prior as a language model, - We'll be somewhat ***satisfied in a scientific sense*** to notice that ***the prior coincides with surprisal, a concept from psycho-linguistics*** (Sec 3.5). They are also shown in "Recipe for Supporting Our ELBO-based Formulations" in Sec 3.

Response to Reviewer E7HT (Comment 11-13)

## Reviewer's Comment 11 (about Sec 3.1; Appendix B.1) > It would also be helpful to give an indication of what from Schulman et al. (2015) is being applied (i.e., the what the "stochastic computation approach" is). > As a result, I'm not convinced that the reconstruction game, absent modifications to the traditional object (e.g., length penalty), assumes a uniform prior of messages. Thank you for pointing it out. Apart from the standard VAE, we adopted the REINFORCE-like optimization but the "reward" (reconstruction term) is differentiable (w.r.t the receiver $\theta$). The differentiability of the "reward" hinders the direct application of REINFORCE. That is why, instead of Williams (1992), we mentioned Schulman et al. (2015), who proposed some sort of the generalization of REINFORCE. As you pointed out, our proof would become clearer by explicitly explaining how to utilize the result of Schulman et al. (2015). ## Revision for Comment 11 To clarify the proof, we added two lemmas (Lemma 1; 2) in Appendix B.1 to support the proof description (The lemmas can be seen as a special case of Schulman (2015) and/or REINFORCE). ## Reviewer's Comment 12 (about Sec 3.1; Appendix B.1) > It seems like the $P(m)\sim\mathop{\mathbb{E}}\nolimits_{x\sim P_{obj}}[S(m|x)]$ should be the prior over messages. We understand your concern, but this is not the case. We hope that the above revision made our proof persuasive enough to regard the uniform distribution as the prior. Let us explain why the average sender is not the prior in the conventional EC signaling game, using some equations. First, let us decompose Eq. 9 as follows: $$ \begin{aligned} \nabla_{\phi,\theta}\mathop{\underbrace{\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log R_{\theta}(x|m)]}}\limits_{\text{conventional objective}} &\mathop{\underbrace{=}}\limits_{\text{proof target}} \nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log R_{\theta}(x|m)+\log P_{\textrm{unif}}^{\textrm{prior}}(m)] \\\\ &= \mathop{\underbrace{\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log R_{\theta}(x|m)]}}\limits_{\text{same as the left-hand side.}} +\mathop{\underbrace{\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log P_{\textrm{unif}}^{\textrm{prior}}(m)].}}\limits_{\text{should be proven $=0.$}} \end{aligned} $$ Based on this decomposition, what we have to prove essentially is: $\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log P_{\textrm{unif}}^{\textrm{prior}}(m)]=0$. According to Lemma 1 (which is added in the revised version), it can be further transformed as follows: $$ \begin{aligned} \nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log P_{\textrm{unif}}^{\textrm{prior}}(m)] &\mathop{\underbrace{=}}\limits_{\text{From Lemma 1.}} \mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\nabla_{\phi,\theta}\log P_{\textrm{unif}}^{\textrm{prior}}(m)+(\log P_{\textrm{unif}}^{\textrm{prior}}(m))\nabla_{\phi,\theta}\log S_{\phi}(m|x)]. \\\\ &= \mathop{\underbrace{\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\nabla_{\phi,\theta}\log P_{\textrm{unif}}^{\textrm{prior}}(m)]}}\limits_{(\dagger)} +\mathop{\underbrace{\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[(\log P_{\textrm{unif}}^{\textrm{prior}}(m))\nabla_{\phi,\theta}\log S_{\phi}(m|x)]}}\limits_{(\ddagger)}. \end{aligned} $$ Here, - Term $(\dagger)$ would be $0$ if $\nabla_{\phi,\theta}\log P_{\textrm{unif}}^{\textrm{prior}}(m)=0$. - Term $(\ddagger)$ would be $0$ if $\log P_{\textrm{unif}}^{\textrm{prior}}(m)$ is constant w.r.t $m$ (See Lemma 2, which is added in the revised version). Thus, we should define $P_{\textrm{unif}}^{\textrm{prior}}(m)$ as a uniform distribution, i.e., $P_{\textrm{unif}}^{\textrm{prior}}(m)=\frac{1}{|\mathcal{M}|}$. Now, let us assume a prior distribution to be $S_{\phi}(m):=\mathop{\mathbb{E}}\nolimits_{x\sim P_{\textrm{obj}}}[S_{\phi}(m|x)]$ as you suggested. Then, what we have to check (prove/disprove) is whether $\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log S_{\phi}(m)]=0$. But the gradient is not zero in general, since $$\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log S_{\phi}(m)]=-\mathop{\mathbb{E}}\nolimits_{S_{\phi}(m)}[-\log S_{\phi}(m)]=-\mathcal{H}(S_{\phi}(M)).$$ ## Reviewer's Comment 13 (about Sec 3.2) > Sec 3.2: what is a "heuristic variant of [a] VAE"? We just meant by that phrase that the conventional EC signaling game is similar to VAE. Indeed, it might be an awkward phrase as you imply. ## Revision for Comment 13 We rephrased the phrase as follows: > (***Revision***) In this sense, the conventional signaling game is similar to VAE.

Response to Reviewer knBD

We greatly appreciate your positive feedback on our research! > Based on the current formulation, it seems that the distractors on the receiver’s side are not considered. > How would you incorporate the context of the distractors and their corresponding influences into the prior design? This is an important point. Thank you for highlighting it. Similar to Lewis's signaling (reconstruction) game, Lewis's discrimination (referential) game has also been a significant model in EC. Due to the generative nature of our storytelling, we have focused on the reconstruction game in this study. We plan to extend our study to include the discrimination game in our future work.

Response to Reviewer unmd

Thank you for reviewing our paper. ## Reviewer's Comment 1 > The paper is hard to read. The theoretical section includes symbols and equations without full explanation. Thank you for pointing out the need for clearer explanations in the theoretical section. Taking all the reviewers' comments into consideration, we have now added explanations for symbols and equations. If you have any other specific problems or comments, please do not hesitate to mention them. ## Reviewer's Comment 2 > The experiments are weak. The compared methods lack descriptions, and the performance improvement is not well explained. We appreciate your feedback regarding the experimental section. We believe we have adequately addressed the important aspects of the theoretical contribution through mathematical proof. Regarding word length and segmentation, we are confident that the experimental results we have obtained support the main claims of our contribution. However, it is possible that we may have overlooked some points that the reviewer has noticed. If you believe there is a lack of experiments in terms of supporting specific hypotheses, we would greatly appreciate your feedback. ## Reviewer's Comment 3 > The studied problem lacks of enough audience. We are afraid that you may be underestimating the growing interest in Emergent Communication (EC) and Representation Learning. Both of them are increasingly being recognized as important in recent years, as evidenced by the significant number of papers published in major machine leaning conferences. Our paper bridges these two areas by demonstrating how EC signaling game can be re-interpreted as representation learning in generative models, namely beta-VAE.

Studied problem does have an audience

I would explicitly disagree with Weakness (3) given by Reviewer unmd. I think theoretical analysis of emergent communication games is much needed in the field as a whole. For the most part, EC works lean towards "engineering" approaches (e.g., tinkering with empirical evaluation) instead of theoretical analysis that allows for principled predictions before empirical evaluation (even if they turn out to be wrong). This allows for iterative development of theoretical models as they produce predictions which are subsequently tested and point to revisions needed in the theoretical model. For this reason, I definitely think that the overall direction of this paper is important.