Transformers predict LCG sequences in-context for fixed moduli up to 2^32 and unseen moduli up to 2^16 by learning the modulus factorization and digit-wise periodic structure.
Transformers learn to implement preconditioned gradient descent for in-context learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
(How) Can Transformers Predict Pseudo-Random Numbers?
Transformers predict LCG sequences in-context for fixed moduli up to 2^32 and unseen moduli up to 2^16 by learning the modulus factorization and digit-wise periodic structure.