To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models

TL;DR

State Space Models (SSMs) have become the leading alternative to Transformers for sequence modeling tasks. Their primary advantage is efficiency in long-context and long-form generation, enabled by fixed-size memory and linear scaling of computational complexity.

Opening excerpt from the authors’ abstract. source