Recurrent Transformers Retain a Global Workspace, but Access It Differently

Summary

A new arXiv study finds that looped and depth-recurrent transformers can form global-workspace-like representations, but recurrence changes how those representations are accessed. Ouro-2.6B rebuilds workspace content in every loop, while Huginn-0125 carries content across sixteen recurrences but limits interventions to a short sliding window.