Feedforward transformers push each state update into deeper layers, exhausting the model's depth, so the authors argue step-wise recurrence is required and propose a taxonomy of recurrent transformer designs.
Towards optimal adapter placement for efficient transfer learning
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
representative citing papers
GD-FPS is a gradient-free, forward-pass-only parameter selection method for PEFT that identifies important weights by scaling magnitudes with relative activation growth against a pre-training anchor, matching or beating gradient-based baselines on 26 visual tasks while cutting memory by ~18x and run
citing papers explorer
-
The Topological Trouble With Transformers
Feedforward transformers push each state update into deeper layers, exhausting the model's depth, so the authors argue step-wise recurrence is required and propose a taxonomy of recurrent transformer designs.
-
GD-FPS: Growth-Driven Feedforward Parameter Selection for Efficient Fine-Tuning
GD-FPS is a gradient-free, forward-pass-only parameter selection method for PEFT that identifies important weights by scaling magnitudes with relative activation growth against a pre-training anchor, matching or beating gradient-based baselines on 26 visual tasks while cutting memory by ~18x and run