pith. sign in

For Llama-3.2-1B on theFineWeb-Edusubset, we follow the hyperparameter configuration of prior work and apply the same search protocol to all baselines, including ours

while limiting additional hyperparameter tuning, we fix the learning rates of key, value parameters to the maximum learning rate without scheduling, set their weight decay to zero · 2025

1 Pith paper cite this work. Polarity classification is still indexing.

1 Pith paper citing it
browse 1 citing papers

fields

cs.LG 1

years

2025 1

verdicts

UNVERDICTED 1

representative citing papers

MIDUS: Memory-Infused Depth Up-Scaling

cs.LG · 2025-12-15 · unverdicted · novelty 7.0

MIDUS replaces duplicated FFN branches in depth up-scaling with head-wise memory layers using product-key retrieval and HIVE to deliver lightweight, head-conditioned residual capacity.

citing papers explorer

Showing 1 of 1 citing paper.

  • MIDUS: Memory-Infused Depth Up-Scaling cs.LG · 2025-12-15 · unverdicted · none · ref 32

    MIDUS replaces duplicated FFN branches in depth up-scaling with head-wise memory layers using product-key retrieval and HIVE to deliver lightweight, head-conditioned residual capacity.