Pith. sign in

F or each coordinate with dt,i > 0, γt,i = 1 1 + exp ( −a log dt,i ǫgate ) (25) = 1 1 + (ǫgate/dt,i)a (26) = d a t,i d a t,i +ǫ a gate

14 A G ATEDADAM W’ S GATE WITHOUT AN EXPLICIT SIGMOID The sigmoid-of-log gate has an equivalent power-law form · 2017

1 Pith paper cite this work. Polarity classification is still indexing.

1 Pith paper citing it
browse 1 citing papers

fields

cs.LG 1

years

2026 1

verdicts

CONDITIONAL 1

representative citing papers

Training nGPT

cs.LG · 2026-08-02 · conditional · novelty 6.0

An nGPT training recipe with new optimizer and normalization components cuts pretraining tokens roughly in half for a 14B hybrid mixture-of-experts language model compared with an AdamW baseline.

citing papers explorer

Showing 1 of 1 citing paper.

  • Training nGPT cs.LG · 2026-08-02 · conditional · none · ref 12

    An nGPT training recipe with new optimizer and normalization components cuts pretraining tokens roughly in half for a 14B hybrid mixture-of-experts language model compared with an AdamW baseline.