A Workflow for Understanding LLM Architectures

Summary

Sebastian Raschka describes how he creates architecture sketches for large language models. He usually begins with official technical reports, but notes that recent papers, especially for open-weight industry models, often disclose fewer architectural details. When weights are available on Hugging Face and the model is supported by the Python Transformers library, he examines the configuration file and reference implementation directly. The code can reveal concrete architecture details that are missing from the paper. The workflow is mainly intended for open-weight models and does not apply well to proprietary systems such as ChatGPT, Claude, or Gemini. Raschka keeps the process deliberately manual because inspecting several models by hand is useful for learning how their architectures work.