Error-Driven Prompt Optimization for Arithmetic Reasoning

Recent advancements in artificial intelligence have sparked interest in industrial agents that support analysts in regulated sectors, such as finance and healthcare, within tabular data workflows. A key capability for such systems is performing accurate arithmetic operations on structured data while ensuring sensitive information never leaves secure, on-premises environments. Here, we introduce an error-driven optimization framework for arithmetic reasoning that enhances a Code Generation Agent (CGA), specifically applied to on-premises small language models (SLMs). Through a systematic evaluation of leading SLMs (Qwen3 4B, Gemma 3n E4B, Gemma 3 4B), we find that while the base model exhibits fundamental limitations in arithmetic tasks, our proposed error-driven method, which clusters erroneous predictions to refine prompt-rules iteratively, significantly improves performance, elevating the model’s accuracy to 66.45%. Our results suggest that developing reliable and interpretable AI assistants with potential for industrial deployment can be achieved not only through costly fine-tuning but also through systematic, error-driven prompt optimization, enabling small models to surpass larger language models (e.g., GPT-3.5 Turbo) in a privacy-compliant manner.