How to Build a Local Coding Agent with Open-Weight Models

Summary

Sebastian Raschka presents a detailed tutorial for building a local coding-agent workflow from open-source tools and open-weight language models. The setup separates the locally served model from the coding harness: Ollama provides the inference server, while Qwen Code, Codex or Claude Code supplies file access, editing, command execution and verification. He focuses on Qwen3.6 35B-A3B with Qwen Code, while also discussing North Mini Code, Gemma 4 E2B and Nemotron 3 Nano. Qwen3.6 35B-A3B requires about 22 GB to download and roughly 30-40 GB of RAM; in the reported tests, Qwen3.6 and North Mini Code generated about 40 tokens per second on a recent Mac Mini and 30 tokens per second on a DGX Spark with long contexts, although macOS memory measurements were imperfect. A small tool-reasoning test gave Qwen3.6 3/5, North Mini Code 1/5 and Gemma 4 E2B 0/5, while a later five-task agent assessment gave Qwen3.6 and North Mini Code 4/5 and showed that Gemma was substantially weaker. The author recommends comparing models with personal tasks as well as changing benchmarks because benchmark scores and weighting can shift. The article also stresses that a coding harness has access to files and shell commands, so users should audit installation scripts, permissions, secret handling, network calls, telemetry, integrations and prompt-injection surfaces, and should use a separate machine, account or virtual environment for experiments. Qwen Code can be configured to use Ollama’s local OpenAI-compatible endpoint, while Codex can use a separate Ollama profile and Claude Code can be launched through Ollama. The author notes that local prompts may remain on the machine, but Qwen Code can still transmit usage metadata unless privacy and telemetry settings are disabled. In the reported comparison, Codex used fewer tokens than Claude Code, while task correctness depended on both the model and harness. The article concludes that local models in the 30-35B range are capable for many coding tasks, but reliability, permissions and evaluation matter as much as installation convenience.