Sequences of Logits Reveal the Low Rank Structure of Language Models
ICLROral2026
TL;DR
We exploit the low-rank structure of the logit matrices of LLMs to draw new empirical and theoretical conclusions.
Opening excerpt from the authors’ abstract. source
Read the paper
Topics
language model llm