Response to Reviewer HVvH
We thank you for your insightful comments on improving our paper. We hope the following could address your concerns.
> **The method is currently only tested in relatively simple MPE environments**
MPE environments emphasize different interaction types: competitive, cooperative, and mixed, fostering diverse agent dynamics. In future work, we plan to apply our methods to other environments, such as StarCraft. In terms of agent number, we already conduct a task with 8 agents in a 4x4 grassland setting. Additionally, we extended our analysis to a 10x10 grassland test, with the results presented below.
> **As the number of agents increases, the computational complexity of inferring attention weights for multiple agents may become a bottleneck. The paper does not fully address how the method scales with a larger number of agents or more diverse agent types.**
The inverse-att network will scale up in the benefit of the attention structure. We present the results for 20 agents in the Grassland environment with a 10-10 scale. The Inverse-Att agents not only scale effectively but also outperform other ablations and baseline methods.
| | MAPPO | IPPO | MAA2C | ToM2C | Self-Att | Inverse-Att |
|---------------------|---------|---------|---------|---------|----------|-------------|
| **Grassland Sheep** | -68.15 | -74.64 | -55.00 | -58.96 | -143.50 | -52.34 |
| **Grassland Wolf** | 90.47 | 49.81 | 30.50 | 30.21 | 120.08 | 132.39 |
> **The method assumes that agents are of the same type, which limits its generalization.**
We do not assume that all agents are of the same type. For instance, in a competitive environment like Adversary, there are two distinct types of agents: sheep and wolves. Since inverse attention employs a goal-focused (GF) representation, this mechanism can be applied to all types of teammates, regardless of their roles.
> **The inverse attention mechanism is trained in an offline manner, which may limit its adaptability in a rapidly changing environment.**
Inverse attention is initially trained offline to abstract the relationships between agents' observations and actions. However, an online training phase is introduced later (as described in the second while loop of Algorithm 1), where MARL is applied to adapt more effectively to other agents and dynamic environments. Experimental results in Tables 1–5 demonstrate that, even when interacting with various unseen agents or human players, inverse attention consistently outperforms all other baseline methods.
> **Does the performance of inverse attention agents degrade as the number of agents increases, particularly in larger environments like the 4-4 Grassland game?**
The 4-4 and 10-10 results show that inverse-att agents outperform the other baselines with a large margin which could scale up as the agent number increases.
> **Why does the cooperative gain between multiple inverse attention agents not increase linearly?**
Yes, we believe cognitive loops or mutual inference inaccuracies indeed plays a role in the absence of linear reward increase.
> **Why do inverse attention agents perform inconsistently when cooperating with humans in some roles (e.g., sheep in Adversary), and how can this cooperation be made more robust?**
We did not observe any inconsistencies in the behavior of Inverse-Att agents, including the sheep in the Adversary environment. When comparing them to other methods, Inverse-Att agents consistently achieve much higher rewards with relatively low variance compared to all other baselines. Could you clarify what you mean by "not robust"?
> **How is the accuracy of the inverse attention network related to task success, and why doesn't high inference accuracy always correlate with better performance in tasks like Adversary?**
The higher accuracy of the inverse attention network allows agents to achieve better performance by reducing the noise in their input. This advantage applies across all tasks, including competitive scenarios like Adversary. For instance, if Agent A knows that teammates B and C are moving left to avoid a wolf, it might decide to go right, avoiding the crowd and reducing its risk of being caught.