Rebuttal Part 2
**[W3: Contribution of each FINCON component/ Ablation of memory module.] We have added an ablation study to assess the effectiveness of FINCON's episodic memory design.**
1. To clarify the contribution of each part, we provide our detailed motivation as follows:
- Multi-agent setup: We employ a unique Manager-Analyst hierarchical multi-agent structure, where each agent is responsible for a specific task. This synthesized approach enables collaborative tackling of sequential decision-making challenges in complex financial markets. The hierarchical communication structure enhances efficiency by eliminating redundant peer-to-peer interactions, thereby optimizing resource utilization.
- Reinforcement learning: The actor-critic structure in classical RL serves as an inspiration for our system design, though FinCon adapts its core principles rather than applying them directly. The actor component in our system is tasked with selecting policies. The critic component in our system does not estimate value functions as in traditional RL,it is designed to reflect on and learn from past actions by evaluating their outcomes based on feedback from the environment.
- Conceptual Verbal Reinforcement (CVR): The CVR helps to update manager's investment beliefs over multiple training episodes. The ablation study in **[W3.2]** below further illustrates its contribution to FINCON's performance.
- Convex optimization: Convex optimization is employed exclusively to determine asset allocation in portfolio management tasks, and we leverage this method for its proven reliability and effectiveness in portfolio management. Our work is pioneering in its integration of LLMs for managing portfolios, marking the first instance of such an application in the field.
These mechanisms work in concert to ensure FINCON's high-quality decision-making capability from different perspectives. Furthermore, we believe the setup of FINCON maintains novelty from two fronts: **Empirical novelty**: FINCON addresses the complexity of volatile market dynamics and achieves state-of-the-art performance in various practical financial decision-making tasks. Moreover, it is the first financial language agent system to support portfolio management functionality.
**Technical novelty**: FINCON introduces the unique CVR mechanism for multi-agent system risk control. This represents a novel development beyond verbal reinforcement, designed specifically to synthesize hierarchical multi-agent collaboration.
2. Our memory module is composed of working memory, procedural memory, and episodic memory. *While the first two components support essential functions for each agent's basic operations, the episodic memory refers to the updated investment beliefs through FINCON's CVR mechanism, continuously enhancing its decision-making capabilities over episodes.* Therefore, conducting an ablation study on episodic memory is more meaningful. Our experimental outcomes are summarized in the same **Table C in W2**. Given the same choice of training and test periods as Table A and B, the results demonstrate that CRs and SRs have been substantially enhanced after iterating over four training episodes of investment belief updates. This conclusion holds true for both single stock trading and portfolio management tasks, and remains robust across various market conditions (bearish, bullish, and mixed).
**[W4: Reproducibility] We have shared the source code of FinCon with standard protocol through AC.**
Please don't hesitate to request it from the AC.
**[W5: Citation] We will add this paper as part of our related work and citation list.**
We agree the significant position of this paper in the field of textual optimization. We will incorporate it in the related work. And, the paper will also be properly cited in our reference list.
**[Q1: CVR vs. OPRO] Differences Between CVR and OPRO are explained below.**
1. OPRO includes the entire optimization trajectory in meta-prompts, while CVR selects segments of consistent profit or loss.
2. Unlike OPRO, which only specifies the upgrade direction, CVR mimics the learning rate using the overlapping rate between conceptualized insights.
3. OPRO handles deterministic problems like linear regression and the traveling salesman problem, as well as manually annotated datasets (e.g., GSM8K [1], BBH [2]). In contrast, CVR addresses stochastic optimization in financial investment, managing the environment's randomness and complexity.
[1] Cobbe, Karl, et al. "Training verifiers to solve math word problems." arXiv preprint arXiv:2110.14168 (2021).
[2] Suzgun, Mirac, et al. "Challenging big-bench tasks and whether chain-of-thought can solve them." arXiv preprint arXiv:2210.09261 (2022).