Transferring Linear Features Across Language Models With Model Stitching

TL;DR

In this work, we demonstrate that affine mappings between residual streams of language models is a cheap way to effectively transfer represented features between models…

Opening excerpt from the authors’ abstract. source