LIG: Layer-wise Integrated Gradients for Within-Layer Flow Analysis in Transformers

Set-to-set Integrated Gradients at Attention and MLP module boundaries inside each Transformer layer.

Eight Suzuki1,3 · Hideitsu Hino1,2 · Noboru Murata1

  1. 1 Waseda University
  2. 2 The Institute of Statistical Mathematics
  3. 3 Fujitsu Limited
Papersoon arXiv Code Project page Contact

Within-layer token contributions (z→z)

Sample
00410
Route
Direct LIG (layer z→z)
Baseline
Zero

Input sentence

The firm's drop in net reflected weaker revenue in transactions for its own account — a decline of 19% to $314.6 million on reduced revenue from trading fixed-income securities.

How to read

Within-layer token-to-token contributions on the layer-direct z→z map (token vectors z at layer input — see the project page notation). Each column is a target token position; circles show source → target contribution in that layer.

  • Left label on each box: target token (receives the contribution)
  • Top / bottom token labels (horizontal): source token — circle position shows who contributed
  • Vertical axis: layers 0–11 (bottom row = layer 0, top row = layer 11; lower layers are closer to the input)
  • Circle size: contribution from source → target within the same layer
  • Path thickness: inter-layer contribution paths (toggle with "Show inter-layer contribution paths" in Display options)
  • Red highlight: clicked target token

Display options

Green = pivot at 100/N (drag ↕ to bend). Orange = left of pivot, blue = right — drag each handle independently. Dashed line marks 100/N.

Layout