Open-World AI Requires Inference-Time Learning and Rich, Disentangled Representations

Summary

The paper distinguishes closed-world AI, which targets predefined tasks with independently sampled data, from open-world AI, which must handle tasks not known in advance, often from few examples and with limited human-designed prior knowledge. It argues that open-world systems need inference-time learning because tasks arrive during use and their data is grouped by related tasks rather than randomly shuffled. By modeling tasks as sharing latent factors and decomposing the training objective, the authors derive two additional requirements: a rich representation that preserves more information than ordinary iid generalization requires, and predictive disentanglement that aligns related low-dimensional input and output groups. The analysis also predicts that systems need a consistent inner learner and may benefit from long contexts spanning related tasks. Experiments on language models with up to 8 billion parameters assess whether these properties exist, rather than ranking models by a single performance score. Models meeting the proposed requirements improve as they receive more data from related tasks, while models that do not meet them degrade. Increasing model scale or interleaving layers of different types did not repair the missing properties in the reported tests.