From Markov to Laplace: How Mamba In-Context Learns Markov Chains

ICLROral2026

Authors: Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva
Affiliation: EPFL - EPF Lausanne
Venue: ICLR 2026
Track: Oral

TL;DR

We uncover an interesting phenomenon where a single-layer Mamba represents the Bayes optimal Laplacian smoothing estimator when trained on Markov chains and we demonstrate it theoretically and empirically.

Opening excerpt from the authors’ abstract. source

Read the paper

OpenReview

Topics

mamba

← All ICLR 2026 Oral papers · Browse the whole archive