OptChat proposes an endlessly persistent AI-agent chat with a hierarchical memory tree

Summary

OptChat is a technical specification for an AI-agent chat whose history never ends and is never deleted. It stores every user message, agent reply, tool call, tool result, and imported note verbatim in append-only JSONL logs, while a background compactor builds a binary tree of summaries. Each tree node covers either one message or two adjacent child nodes and targets a 512-byte summary. The agent does not carry a conversation forward between turns; each turn starts a fresh model call with a fixed-size view of the whole chat, followed by the new user message. Recent history appears at finer resolution, while older history is represented by progressively coarser summaries. When a summary is too vague, the agent can use a zoom tool to open it recursively until it reaches the original message. The design waits for all visible parts to be summarized before starting a turn, so the model never acts on truncated text. Its view is folded incrementally under a 128,000-byte budget: adjacent, compatible summary nodes are merged according to an age-and-size rule, and already merged regions are never split. This preserves a stable prefix between turns and improves prompt-cache reuse. The specification also defines strict compactor ordering, retries, UTF-8 size handling, fsync-based durability, single-writer locking, capped tool results, and optional subagents. It recommends keeping system prompts, tools, and cache breakpoints stable, while avoiding timestamps and other volatile prompt content. The approach is intended to reduce context rot, preserve user instructions and decisions, and make long-running agent work searchable without relying on a separate memory tool. The document describes a working implementation pattern rather than reporting a comparative evaluation or measured model-quality study.