Sifthound Offers a Self-Hosted, Tavily-Compatible API for AI Agents

Summary

Sifthound is an MIT-licensed, open-source web search and extraction API that teams can run on their own infrastructure for AI agents and LLM applications. It reproduces Tavily's /search, /extract, /crawl, and /map endpoints with matching request and response shapes, allowing the official tavily-python SDK and LangChain's langchain-tavily integration to work by changing the API base URL. Search uses a SearXNG metasearch instance and does not require a search API key; extraction uses trafilatura to return clean Markdown or text, while ranking combines BM25 relevance with the upstream search order. Advanced search fetches pages and returns relevant chunks, and optional answers are written by Claude from retrieved results. The project also supports site crawling and URL mapping with depth, breadth, limits, and path or domain filters. Sifthound exposes four read-only MCP tools over HTTP or stdio for Claude Code, Claude Desktop, Cursor, and other MCP clients. Its security controls block private, loopback, and link-local addresses, including through redirects and DNS rebinding checks, when fetching caller-supplied URLs. The project has been tested with tavily-python 0.8.4 and langchain-tavily 0.2.18. It can be deployed with Docker, pip, or a published image for Linux amd64 and arm64. Compared with hosted Tavily, it lacks neural reranking and the research endpoint, and it does not render JavaScript pages.