A Guide to the Open-Source AI Stack for Coding Agents

Summary

Together AI presents the open-source AI stack as a set of independent layers that developers can combine and replace as their needs change. The stack includes the model, the inference provider, gateways or routers, the harness that manages conversations and tool use, and extensions such as skills and Model Context Protocol (MCP) servers. The article explains that open-model adoption does not require training models or operating a large GPU cluster: application developers can use hosted inference through APIs and select among many models. Large models generally handle ambiguity, multi-step reasoning, broad codebase context, and varied tasks more reliably, while smaller models can match them on narrowly specified work and offer lower latency and cost. The article uses Kimi K3 as an example of a large model and GLM 5.3 Flash as an example of a smaller model, citing their total and active parameter counts and an approximate cost comparison. It recommends treating model choice as tool selection and evaluating models against real workloads rather than relying only on aggregate leaderboards. Cloud inference providers run models on their GPUs and charge by input and output tokens; gateways aggregate multiple providers, allow routing between open and closed models, and can help compare price and latency. A harness sits between the user, model, and codebase: the model decides which searches or commands are needed, while the harness executes tools, returns context, applies patches, manages permissions, and recovers from failures. The article also describes terminal, editor, web, and desktop harnesses, plus reusable skills and MCP servers for connecting agents to external data and tools. Because long conversations accumulate files, logs, failed approaches, and stale assumptions, it recommends starting fresh sessions between tasks or model changes. One suggested workflow assigns planning, implementation, and review to different models, using larger models for open-ended planning and review and smaller models for focused implementation. The central design principle is composability: developers can change models, providers, routers, harnesses, and tools independently, compare cost and latency on real tasks, and build a custom AI coding stack instead of committing to one product ecosystem.