First Attentions Last: Better Exploiting First Attentions for Efficient Transformer Training.

Wed, 01 Oct 2025 01:00:00 +0200

Gyudong Kim, Hyukju Na, Jin Hyeon Kim, Hyunsung Jang, Jaemin Park, Jaegi Hwang, Namkoo Ha, Seungryong Kim, Young Geun Kim:
First Attentions Last: Better Exploiting First Attentions for Efficient Transformer Training. CoRR abs/2510.14614 (2025)