Characterizing the Discrete Geometry of ReLU Networks

TL;DR

We describe the geometry of the polyhedral complexes defined by the linear regions of ReLU networks, both by theoretically bounding their connectivity and diameter and by empirically characterizing it using experiments on trained networks.

Opening excerpt from the authors’ abstract. source