Study Finds LLM Watermarking Can Alter AI Agent Behavior

Summary

A Lasso Security study examines whether SynthID-Text, a text watermarking method used to identify AI-generated content, changes the behavior of models used inside AI agents. The method modifies token selection during generation while preserving the original token distribution in expectation, so a fixed watermark key can still produce different outputs. In paired tests using BFCL v4, HarmBench, and JailbreakBench, watermarking changed tool-call verdicts across 21 model-temperature combinations, with average paired disagreement of 6.5%; at temperature 1.0, disagreement reached 16.8% for phi-4 and 9.9% for Llama-3.1-8B. Accuracy losses could conceal larger item-level changes, including wrong tools, incorrect arguments, or malformed output. Watermarking also changed refusal behavior, particularly when harmful prompts included a fixed prompt-injection technique. For example, at low temperature, gemma-3-27b refusal-related churn rose from 6.0% on bare harmful requests to 23.5% under injection, while net compliance increased by 12.5 points. Effects varied substantially by model and watermark key, sometimes changing direction. The study measures model-level refusal rather than complete end-to-end agent failures, but argues that altered refusals and tool calls can affect agents with access to actions. It recommends repeating paired evaluations and red-teaming under the exact watermark configuration intended for deployment. The authors distinguish provenance detection from behavioral stability: unchanged expected text quality does not guarantee unchanged safety or tool-use behavior.