FLIPS identifies LLM instances with 96% closed-set and 90% open-set accuracy by exploiting biases in generated binary random sequences across 237 instances.
Deterministic or probabilistic? the psychology of LLMs as random number generators.arXiv preprint arXiv:2502.19965
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
years
2026 4verdicts
UNVERDICTED 4representative citing papers
Language models have an intrinsic randomness floor: transformers show ~0.30 entropic deviation from uniform on neutral prompts, accounting for 88-93% of observed non-randomness, while state-space models exhibit twice the deviation and strong temperature sensitivity.
A standardized evaluation framework shows comparative bias tests in LLMs activate more latent discrimination than isolated assessments, worsened by CoT and scaling with model size.
A 448-problem benchmark and KS@N metric show LLMs rarely match target distributions, with no model exceeding 40% at KS@100.
citing papers explorer
-
FLIPS: Instance-Fingerprinting for LLMs via Pseudo-random Sequences
FLIPS identifies LLM instances with 96% closed-set and 90% open-set accuracy by exploiting biases in generated binary random sequences across 237 instances.
-
The Randomness Floor: Measuring Intrinsic Non-Randomness in Language Model Token Distributions
Language models have an intrinsic randomness floor: transformers show ~0.30 entropic deviation from uniform on neutral prompts, accounting for 88-93% of observed non-randomness, while state-space models exhibit twice the deviation and strong temperature sensitivity.
-
To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias
A standardized evaluation framework shows comparative bias tests in LLMs activate more latent discrimination than isolated assessments, worsened by CoT and scaling with model size.
-
UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs
A 448-problem benchmark and KS@N metric show LLMs rarely match target distributions, with no model exceeding 40% at KS@100.