Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth

Attention-based architectures have become ubiquitous in machine learning, yet\nour understanding of the reasons for their effectiveness remains limited. This\nwork proposes a new way to understand self-attention networks: we show that\ntheir output can be decomposed into a sum of smaller terms, each involving the\noperation of a sequence of attention heads across layers. Using this\ndecomposition, we prove that self-attention possesses a strong inductive bias\ntowards "token uniformity". Specifically, without skip connections or\nmulti-layer perceptrons (MLPs), the output converges doubly exponentially to a\nrank-1 matrix. On the other hand, skip connections and MLPs stop the output\nfrom degeneration. Our experiments verify the identified convergence phenomena\non different variants of standard transformer architectures.\n