Its better for high-level debugging, clearly shows flow magnitudes and which heads contribute most. Standard heatmaps still win for fine-grained token-to-token patterns. but on larger models it gets dense/cluttered fast (even with collapsing)
Technical analysis of attention head visualization and model interpretability
By
–