Learning long range dependencies through time reversal symmetry breaking

TL;DR

We propose a backward-mode AD proxy using only forward passes applying to Hamiltonian recurrent units and stacks thereof (namely, SSMs) with theoretical guarantees and experimental evidence…

Opening excerpt from the authors’ abstract. source