Formalizes four concurrency anomalies in multi-agent LLM systems and mechanically verifies a hierarchy of sound detectors and preventions realized in Rust runtimes using TLA+ and Verus.
The good, the bad, and the greedy: Evaluation of llms should not ignore non-determinism
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
LLMs show implementation-induced randomness even at T=0 that can be characterized as an effective background temperature T_bg estimated via an ideal reference system.
LLMs with two prompting strategies and model validation tools produce mostly syntactically correct, conforming, semantically realistic and diverse instances of UML class diagrams.
Repeated sampling scales problem coverage log-linearly with sample count, improving SWE-bench Lite performance from 15.9% to 56% using 250 samples.
Three LLMs exhibit distinct consistency profiles in repeated exercise prescription generation, with GPT-4.1 producing unique but semantically stable outputs while Gemini 2.5 Flash achieves high similarity through text duplication.
citing papers explorer
-
Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent Large Language Model Systems
Formalizes four concurrency anomalies in multi-agent LLM systems and mechanically verifies a hierarchy of sound detectors and preventions realized in Rust runtimes using TLA+ and Verus.
-
Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models
LLMs show implementation-induced randomness even at T=0 that can be characterized as an effective background temperature T_bg estimated via an ideal reference system.
-
LLM-based Generation of Semantically Diverse and Realistic Domain Model Instances
LLMs with two prompting strategies and model validation tools produce mostly syntactically correct, conforming, semantically realistic and diverse instances of UML class diagrams.
-
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
Repeated sampling scales problem coverage log-linearly with sample count, improving SWE-bench Lite performance from 15.9% to 56% using 250 samples.
-
Cross-Model Consistency of AI-Generated Exercise Prescriptions: A Repeated Generation Study Across Three Large Language Models
Three LLMs exhibit distinct consistency profiles in repeated exercise prescription generation, with GPT-4.1 producing unique but semantically stable outputs while Gemini 2.5 Flash achieves high similarity through text duplication.