Title: The Topological Trouble With Transformers

Source: http://arxiv.org/abs/2604.17121v1

Summary:

This research identifies a fundamental topological limit in the Transformer architecture's ability to track dynamic states due to its feedforward nature, which exhausts model depth as states evolve. It advocates for a shift towards recurrent or continuous-thought architectures as essential primitives for achieving the temporally extended cognition required for advanced AI agents.