Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions

Summary

This study examines whether metadata differences affect how large language models select tools in Model Context Protocol systems, where models rely on names, descriptions, and parameter schemas rather than tool implementations. An analysis of 367,174 MCP tools finds substantial schema variation among functionally similar tools, including cases with similar names and descriptions but different parameter structures. The researchers run controlled interventions across five LLMs and five tasks, changing exactly one factor at a time: server name, tool name, description, or parameter schema. They balance tool positions and use conditional logistic regression to estimate effects independently of placement. Parameter-schema changes produce the largest and most consistent shifts in selection, and models generally prefer separate top-level parameters over grouped or composite forms. Descriptions that explain parameters also tend to increase selection, while name effects are weaker and less consistent. Schema effects depend on the parameter names and on how the request expresses the same information. When only one schema satisfies an explicit requirement, all five models select the compatible tool despite positional preferences. The findings indicate that schema design and tool ordering can change which implementation is invoked even when tools provide equivalent functionality.