Diffusion Language Model Knows the Answer Before It Decodes

TL;DR

Diffusion language models (DLMs) have recently emerged as an alternative to autoregressive approaches, offering parallel sequence generation and flexible token orders. However, their inference remains slower than that of autoregressive models, primarily due to the cost of bidirectional attention and the large number of refinement steps required...

Opening excerpt from the authors’ abstract. source