Bifrost Launches Open-Source Gateway for LLMs, MCP and AI Agents

Summary

Bifrost is an open-source, high-performance gateway for applications built with large language models, MCP tools, and AI agents. It provides one OpenAI-compatible API for more than 23 providers, including OpenAI, Anthropic, AWS Bedrock, and Google Vertex, and supports OpenAI, Anthropic, Google GenAI, LiteLLM, and LangChain integrations. The gateway includes automatic failover, load balancing across providers and API keys, semantic caching, multimodal requests, and an extensible plugin system. Its MCP gateway allows models to use external tools such as filesystems, web search, and databases. Enterprise deployments add private networking, custom security controls, clustering, guardrails, usage tracking, rate limits, budgets, virtual keys, OIDC provisioning, secrets management, Prometheus metrics, tracing, and logging. Users can start the HTTP gateway with NPX or Docker, configure it through a web interface, embed it through a Go SDK, or replace existing provider endpoints with small base-URL changes. In the stated sustained 5,000-RPS benchmark, Bifrost reported 100% request success, 11 microseconds of added overhead on an AWS t3.xlarge, and an average request latency of 1.61 seconds including the provider. The repository is licensed under Apache 2.0.