Lasso Finds AI Watermarks Can Change Agent Decisions

Summary

Lasso Security’s report, “The Provenance Tax,” examines whether text watermarking changes the behavior of AI systems rather than merely adding a detectable signal. It studied Google DeepMind’s SynthID-Text in a non-distortionary configuration, which alters token selection statistically during generation; the technology is used for Gemini text, adopted in a version for some Claude models, and available in Hugging Face Transformers. Because LLMs are probabilistic, small sampling changes can produce different tools, arguments, JSON outputs, or refusal decisions, an effect Lasso calls “sampling drift.” In tests of seven models on the Berkeley Function Calling Leaderboard v4 single-turn AST benchmark, six models scored lower on tool-call tasks and four declines were statistically significant. At temperature 1.0, Phi-4 changed its call/no-call decision on 16.8% of tasks while aggregate accuracy fell 2.87%; Llama-3.1-8B changed decisions on 9.9% of tasks with a 0.87% accuracy decline. Across 21 model-temperature tests, paired disagreement averaged 6.5%. Failure patterns varied: Llama-3.1-8B most often produced wrong arguments or selected the wrong tool, while Phi-4 and Granite-3.2-8B more often produced malformed tool output. Lasso separately tested 200 harmful HarmBench requests, 100 benign JailbreakBench controls, and prompt-injection variants. Watermarking changed refusal behavior even without injection and produced larger differences for several models under injection. Gemma-3-27B showed refusal/compliance changes on 23.5% of injected prompts at temperature 0.001, with net compliance shifting 12.5 points higher; Gemma-3-12B’s churn rose from 7.5% to 11%. Llama-3.1-8B changed behavior on 14% and 17.5% of injected prompts at two temperatures, although its net compliance changes were not statistically significant. Results also varied across 11 watermark keys, with Llama-3.1-8B’s injection success ranging from a 4.5% decrease to a 14.5% increase. The study did not demonstrate a complete exploit chain from watermarking to a dangerous action, but it argues that watermarked models should be tested for consistency, tool reliability, and safety before deployment, especially when providers control the key or inference configuration.