SGAnalog Introduces an Open-Source Silicon Benchmark for AI Circuit Design

Summary

SGAnalog is a benchmark for testing whether AI models learn transferable analog circuit design skills and produce outputs that work under defined process and test conditions. It is built from 273 topologically distinct, human-designed circuits submitted to open-source Tiny Tapeout manufacturing shuttles. Each source is retrieved at the recorded submission revision and processed in a fixed containerized environment; eligible schematics and SPICE netlists are exported from the same source file, while commit dates enable training-cutoff analysis and author testbenches provide sizing context. The benchmark covers schematic-to-netlist transcription and device sizing. On 66 transcription tasks evaluated across seven models, the best model achieves 56.1% exact graph-isomorphism matches, and six models show a sharp decline from the small to medium task tier. For one frontier model, removing author-selected labels lowers exact-match performance but leaves aggregate structural F1 unchanged, indicating that labels may help trace connectivity. On 17 sizing tasks, the leading model converges on all proposals and scores 91.2 out of 100 against the human reference. The two newest Claude models refuse four and 11 of the same sizing prompts, despite transcribing them without objection; a proposal without device sizes receives zero. The different rankings across the two tasks show that they test distinct visual and design capabilities, and that refusal in one model family can reflect policy limits rather than capability alone.