NVIDIA Releases Personal AI Router for Distributed Local Inference

Summary

NVIDIA’s Personal AI Router (PAIR) is a local inference router for compatible computers connected to the same network. It discovers participating nodes, manages supported inference engines, and exposes Ollama-compatible and OpenAI-compatible proxy endpoints for applications and agents. Each independent request is sent to one eligible node based on engine availability, model availability, and current workload; PAIR also prefers nodes that already have the requested model. The system is intended for concurrent local workloads, including multi-agent applications, and keeps prompts and responses on the local network when all configured clients, model sources, engines, and nodes are local. PAIR does not combine GPU memory, create a larger logical GPU, shard a model across machines, or split an in-flight request between nodes. It supports Windows 11, Linux, and macOS on x64 and arm64, with Windows on ARM described as experimental, and provides Windows, Debian, and macOS installers. Ollama and LM Studio are the supported inference engines, but each engine and model retains its own operating-system, GPU, driver, and memory requirements. Users can install an engine and model through the desktop application, test traffic, call the local API, and pair additional machines using a six-digit PIN. PAIR currently uses one scheduling policy combining queued work with a coarse, smoothed GPU-utilization signal; it does not yet account for GPU model, available memory, model warmness, or estimated request cost. The project is licensed under Apache 2.0 and cautions users to review its security guidance before deploying local HTTP endpoints, discovery, and cluster networking on untrusted or shared networks.