Beyond the Parameter Monolith: Reconstructive Memory and Executable Skills for Language Models

Summary

The paper investigates whether a language-model system can separate contextual computation, persistent storage, and exact execution instead of encoding all capabilities in one shared parameter system. It proposes FEM-ASM, a finite-element-method-inspired organization in which independently constructed document states and deterministic executable skills send typed proposals to a shared language-model state, while an explicit residual operator reconciles proposals attached to common interface nodes. The authors emphasize that this is an organizational analogy rather than a claim that language has a physical finite-element formulation, and evaluate it through controlled experiments that include negative results. An attention-free Multi-Mesh prototype learns causal language modeling, but the experiments do not establish competitive general capability. A versioned store holds 52,809 reconstructive memory elements within a budget of about 1.7 billion floating-point values; reconstruction remains incomplete at approximately 75% token accuracy. Support-aware lexical indices make the elements addressable while preserving provenance controls during query construction. In executable arithmetic experiments, positional result observations improve neural rendering compared with repeatedly supplying one global result vector, and replacing the output changes the model’s preferred answer. A bounded attachment demonstration also measures the effect of exposing selected evidence, but does not show that loading an entire multi-billion-value store is useful. Overall, the results support separating storage, execution, and neural coordination, while leaving question-only retrieval, unrestricted answer generation, and end-to-end efficiency unresolved.