Security of LLM Agents: A Case Study Approach

Abstract

As large language models (LLMs) evolve into autonomous agents equipped with tools and communication capabilities, they are increasingly deployed in multi-agent systems (MASs) to perform complex tasks. While these systems offer new levels of functionality and efficiency, their security risks remain underexplored. In this paper, we present a focused security analysis of LLM agents by implementing six representative attacks on two real-world MASs. These attacks expose structural and behavioral vulnerabilities unique to agent-based systems. We also introduce and evaluate defensive mechanisms such as fine-tuned agent behaviors, access control via the NGAC (Next Generation Access Control) standard, and a novel "sanity checker" agent for validating agent outputs. Our findings highlight the urgent need for robust, standardized security frameworks for LLM-based MASs and suggest promising directions for future research in agent-level threat modeling and mitigation.