Pre-trained LLMs on HMM next-token prediction appear to use finite-window Soft n-gram-like learned predictors rather than Bayes-optimal inference, as shown by a new activation-probing and causal-patching pipeline.
Title resolution pending
1 Pith paper cite this work, alongside 21 external citations. Polarity classification is still indexing.
1
Pith paper citing it
21
external citations · OpenAlex
fields
cs.AI 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models
Pre-trained LLMs on HMM next-token prediction appear to use finite-window Soft n-gram-like learned predictors rather than Bayes-optimal inference, as shown by a new activation-probing and causal-patching pipeline.