Text watermarking embeds invisible yet detectable markers in generated content to prevent misuse of Large Language Models (LLMs). Existing methods either require fine-tuning or randomly select tokens from the vocabulary to form a green list, which increases generation likelihood. The former is computationally expensive, while the latter may degrade content quality. To address this, we propose Reinforcement Learning- based Adaptive WaterMarking (RLAWM), a novel framework using Reinforcement Learning (RL) to dynamically generate watermarks without fine-tuning. The watermark agent selects green-list tokens based on content semantics, while the detection agent identifies watermarks by analyzing green word frequency and provides feedback to improve concealment. To preserve quality, Maximum Mean Discrepancy (MMD) measures semantic differences between original and watermarked texts. Direct Preference Optimization (DPO) enables end-to-end training and system stability. On LLaMA2-7B, RLAWM improved TPR@1% by 2.48%, best F1 by 1.25%, and reduced perplexity by 3.45%.
Paper
Full text
AdaptiveWatermark: Dynamic Watermarking for Large Language Models with Agent
Semantic Scholar · Computer Science · 2025
Abstract
Text watermarking embeds invisible yet detectable markers in generated content to prevent misuse of Large Language Models (LLMs). Existing methods either require fine-tuning or randomly select tokens from the vocabulary to form a green list, which increases generation likelihood. The former is computationally expensive, while the latter may degrade content quality. To address this, we propose Reinforcement Learning- based Adaptive WaterMarking (RLAWM), a novel framework using Reinforcement Learning (RL) to dynamically generate watermarks without fine-tuning. The watermark agent selects green-list tokens based on content semantics, while the detection agent identifies watermarks by analyzing green word frequency and provides feedback to improve concealment. To preserve quality, Maximum Mean Discrepancy (MMD) measures semantic differences between original and watermarked texts. Direct Preference Optimization (DPO) enables end-to-end training and system stability. On LLaMA2-7B, RLAWM improved TPR@1% by 2.48%, best F1 by 1.25%, and reduced perplexity by 3.45%.