Autonomous Agents for Collaborative Task under Information Asymmetry

**Discussion 1**: Could you provide more insights and analyses on the memory part of the LLM-based agent? **Reply to Discussion 1**: As mentioned above, the design of Mixed Memory aims at solving the information interaction between humans and agents. Unlike previous multi-agent systems that primarily address knowledge-intensive problems, iAgents emphasize solving information-intensive problems in environments with information asymmetry. The former requires the LLM itself to possess extensive knowledge to decompose complex issues and distribute them to various agents. In contrast, the latter emphasizes that agents can obtain, update, and exchange accurate and objective information in real-time, which is precisely the goal of iAgents memory design. 1. As stated in **lines 151-158** of the paper, Mixed Memory provides two different granularity levels of memory. Distinct Memory offers fine-grained, cross-session, objective and truthful memory retrieval to ensure accurate answers, while Fuzzy Memory provides coarse-grained, session-wise memory retrieval of information summaries, relaxing retrieval query conditions to offer a more comprehensive context. The combination of these two can help agents obtain more accurate, objective, and comprehensive information. 2. Moreover, an effective memory mechanism not only depends on the construction of candidates in memory but also requires reasonable queries. iAgents allows agents to observe previous retrieval queries and results, and by combining infoNav to track the progress of overall collaborative tasks, it enables reactive adjustments to achieve better queries **(lines 159-162)**. 3. The memory format of Mixed Memory aligns with the design of iAgents using dialogues as a means of human information source. It can easily store dialogue information as a structured database or ANN database and can also generalize to other forms of human information. Our latest version of iAgents have integrated the Llama index and file management system, allowing various file formats to be used as human information to construct Mixed Memory, as Mixed Memory is related only to information granularity and retrieval methods, not the format of the information content. Based on the experimental results, the problem setting of the FriendsTV dataset is relatively simple and does not require complex asymmetric environment logic reasoning. Answering questions in the given correct context is not difficult (while the Schedule dataset focuses more on reasoning). Therefore, the positioning information is crucial, which aligns with your judgment. For specific details, please refer to our ablation study in **Section 6.2**. Additionally, from a case-by-case analysis, errors often arise from the agent generating inaccurate queries, leading to the retrieval of incorrect context, which subsequently affects subsequent information interaction and reasoning.

**Discussion 2**: can check it as a reference or provide some analysis between your task and this paper **Reply to Discussion 2**: Based on the framework presented in the paper "A Survey on the Memory Mechanism of Large Language Model based Agents," the following conclusions can be drawn about Mixed Memory: 1. **On Why We Need the Mixed Memory**: **From the cognitive psychology perspective**, iAgents require Mixed Memory because each round of communication involves a ReAct [1] process that combines reasoning and acting, necessitating working memory [2] for support. However, unlike the aim of designing memory capabilities akin to humans for agents to replace humans, our approach focuses on creating a society where agents and humans coexist harmoniously. Agents do not replace humans but serve them, and the value of agents lies in the information possessed by humans. Therefore, the design of Mixed Memory ensures that human information sources are accessed by agents only when they are authorized to do so. **From the self-evolution perspective**, Mixed Memory provides agents with substantial intermediate decision information for their decision trajectories, facilitating subsequent optimization based on feedback. The latest version of iAgents incorporates feedback functionality, allowing humans to rate each collaborative result from iAgents. This feedback, along with the entire trajectory, is stored in a database to enable cross-trials optimization. **From the agent application perspective**, iAgents cannot function without Mixed Memory (unless all information fits within the LLM context, such as small datasets like Schedule and NP). The goal of iAgents is to exchange information to solve problems, and this information resides within Mixed Memory. 2. **On How to Implement the Mixed Memory**: The information sources of Mixed Memory include both Inside-trial Information and External Knowledge. Agents improve their current queries by observing previous queries and results, which falls under Inside-trial Information. The retrieved memory, sourced from human information, constitutes External Knowledge. Mixed Memory takes the form of textual data. Mixed Memory supports operations such as reading, writing, and managing memory. It reads memory through ANN and structured queries, writes memory in real-time when user sends new messages, and supports session-wise summarization and file management operations. 3. **On How to Evaluate the Mixed Memory**: We evaluate the effectiveness of Mixed Memory indirectly by observing the final performance of iAgents on the informative bench. Please refer to **Figure 5b** for our ablation study on Mixed Memory. 4. **On the Limitations & Future Directions of Mixed Memory**: Many perspectives mentioned in the survey align with ours, including advancing the research on **Parametric Memory** rather than relying solely on external memory. iAgents aim to experiment with Parametric Memory in future work due to the significant information loss associated with retrieval-based methods. Issues such as inaccurate queries, chunking, similarity metrics, and embedding models contribute to this loss. From a privacy perspective, we aspire for iAgents to be deployed on each user's device, enabling retrieval results without traversing specific information. Parametric Memory aligns with this need, as the forward pass in a network does not expose readable information. We aim to design components like LLM as Hard Disk or Agent as Hard Disk, which allow information retrieval directly through natural language, eliminating the errors introduced by the query/similarity/topk process. Additionally, Mixed Memory exemplifies **Memory in LLM-based Multi-agent Applications**, where iAgents, combined with the infoNav mechanism, achieve synchronization of information and memory among agents. Finally, **Memory-based Lifelong Learning** is also a goal for iAgents. The information humans possess is not limited to a few conversations required for a task, nor to 128k tokens. We need to further enhance memory design to enable each agent to accurately understand the lifelong information of its human users. [1] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022). React: Synergizing reasoning and acting in language models. [2] Baddeley, A. (1992). Working memory.

**Discussion**: Is it possible to further develop any kind of theoretical analysis **Reply to Discussion**: We are delighted that you are interested in the theoretical foundations and developments of multi-agent collaboration under information asymmetry! Due to that we need pages to introduce new problems, new benchmarks, and new methods, we do not have space in the main body of the paper to provide a detailed theoretical exposition. Instead, we have cited relevant foundational literature and provided brief explanations **(lines 32-34, 79-85, 116-140, 107-110)**. Here, we offer a more detailed introduction to the theoretical foundations of iAgents: 1. iAgents are a class of communicative agents [1,2], modeling the communication between agents as a Markov Decision Process. The agent’s actions consist of generating each utterance in the communication, and the state represents the progress of the current task **(lines 116-140)**. For any given agent, its environment comprises the responses of other agents it is communicating with, which is why information asymmetry arises: each agent could partially observe the environment, as it can only perceive the utterances of other agents not the entirety of the information they possess. 2. Furthermore, we model the agents' communication as a ReAct [3] process **(lines 100-106)**, incorporating reasoning and acting into communicative agents. Thus, like ReAct, the theoretical foundation of iAgents is rooted in cognitive science, including inner speech [4], strategization [5], and working memory [6]. Building on ReAct, iAgents introduces the process of reasoning and acting into two types of information interactions **(lines 107-110)**: interactions between agents and humans and interactions among agents themselves. 3. The above points cover the theoretical foundation of iAgents. As for the issue of information asymmetry, its theoretical basis can be traced to two origins. One comes from the Agent Modeling Agent [7] research in the field of Multi-Agent Reinforcement Learning (MARL), where agents, under the constraints of a partially observable environment, model the intentions of other agents to maximize their own utility despite imperfect information. The other aspect derives from the theory of mind [8] **(lines 32-34)**, where agents learn to model the high-order mental states of other agents. iAgents draw on research from these two fields, proposing not only that agents model other agents but also introducing the infoNav mechanism, which explicitly maintains the communication state between agents which fosters effective collaboration among agents under conditions of information asymmetry. [1] Li, G., Hammoud, H., Itani, H., Khizbullin, D., & Ghanem, B. (2023). Camel: Communicative agents for" mind" exploration of large language model society. [2] Qian, C., Cong, X., Yang, C., Chen, W., Su, Y., Xu, J., ... & Sun, M. (2023). Communicative agents for software development. [3] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022). React: Synergizing reasoning and acting in language models. [4] Alderson-Day, B., & Fernyhough, C. (2015). Inner speech: Development, cognitive functions, phenomenology, and neurobiology. [5] Fernyhough, C. (2010). Vygotsky, Luria, and the social brain. Self and social regulation: Social interaction and the development of social understanding and executive functions. [6] Baddeley, A. (1992). Working memory. [7] Raileanu, R., Denton, E., Szlam, A., & Fergus, R. (2018, July). Modeling others using oneself in multi-agent reinforcement learning. [8] Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind?