Paper Citation Record · LEDGER

Changing Answer Order Can Decrease MMLU Accuracy

As of 9 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 7 inbound Pith citation observations for arXiv:2406.19470.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2406.19470 v2

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 7 of 7 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-09T06:31:02.800959+00:00

measured 7 of 7 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-08T16:11:43.132926Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-20T19:13:40.348932Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation cdbfaff9-978e-4087-96fd-5f1321e991a6 · inbound

RoToR: Towards More Reliable Responses for Order-Invariant Inputs cites this paper.

RoToR: Towards More Reliable Responses for Order-Invariant Inputs Changing Answer Order Can Decrease MMLU Accuracy

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-08T16:11:43.132926Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b6d1f93e-e3dd-4009-9000-70e8e9a4973f · inbound

Benchmarking the Pedagogical Knowledge of Large Language Models cites this paper.

Benchmarking the Pedagogical Knowledge of Large Language Models Changing Answer Order Can Decrease MMLU Accuracy

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-06T23:20:18.169890Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 52594fd5-0e98-41bc-a90e-0ea4c98c651b · inbound

Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead cites this paper.

Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead Changing Answer Order Can Decrease MMLU Accuracy

Reference 29

Resolution: verified exact; arxiv_id, observed 2026-05-19T02:12:55.777888Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 4f8cf4c5-2b88-49df-9087-696db27e308e · inbound

EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages cites this paper.

EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages Changing Answer Order Can Decrease MMLU Accuracy

Reference 1

Resolution: verified exact; arxiv_id, observed 2026-05-15T13:35:51.808802Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 372b4a57-b4d2-44ac-bc76-49c7410da6f5 · inbound

From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs cites this paper.

From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs Changing Answer Order Can Decrease MMLU Accuracy

Reference 10

Resolution: verified exact; arxiv_id, observed 2026-05-11T03:15:55.896876Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 088feab2-b4fc-47ed-975b-e960f681f599 · inbound

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language cites this paper.

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language Changing Answer Order Can Decrease MMLU Accuracy

Reference 4

Resolution: verified exact; arxiv_id, observed 2026-05-20T19:13:40.350702Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 31f1a6c9-2d62-42d4-9011-825f53ad4b26 · inbound

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language cites this paper.

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language Changing Answer Order Can Decrease MMLU Accuracy

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-04T05:10:06.677483Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.