SAFE-Guard: A Safety-Aware Federated Ecosystem for Guardrailing Large Language Models

The widespread deployment of Large Language Models (LLMs) has been accompanied by significant concerns regarding their potential to generate harmful, biased, or unsafe content. While various safety alignment techniques exist, they often lack dynamic adaptability and transparency. This paper introduces SAFE-Guard (Safety-Aware Federated Evaluation and Guardrailing), a comprehensive framework for regulating LLM outputs through a dynamic, learning-based approach. At the core of our framework is a ”Guardrail” model, a specialized LLM trained via Reinforcement Learning (RL) to inspect and act upon user prompts. The Guardrail learns a policy to allow, refuse, or safely rewrite prompts, moving beyond static keyword filters. To continuously improve this Guardrail on diverse and sensitive real-world data, we propose a Trust-Aware Federated Fine-Tuning (TFFT) protocol. This protocol ensures the integrity and accountability of the collaborative fine-tuning process by using a trust metric to weigh contributions from different data silos. Furthermore, we address the critical need for transparency by building a framework to quantify and optimize the tradeoff between the system’s safety performance (effectiveness at blocking harmful content while preserving helpfulness) and the explainability of its interventions. We validate SAFE-Guard on prominent safety benchmarks, demonstrating its superior ability to mitigate harmful generations while maintaining utility, its resilience in a federated setting, and its capacity to provide explainable safety controls.