Paper Citation Record · LEDGER

Leveraging LLM Inconsistency to Boost Pass@k Performance

As of 22 August 2026, this Paper Citation Record lists 47 of 47 outbound references and 1 inbound Pith citation observation for arXiv:2505.12938.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2505.12938 v2

Coverage vector

measured 47 of 47 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-15T20:30:13.206926Z

measured 48 of 48 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-22T06:32:14.747728+00:00

measured 1 of 1 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-05-18T10:04:39.223895Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-18T10:06:13.816265Z

Reference resolution

47 of 47 outbound references displayed

verified exact0
verified fuzzy25
unresolved22
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 001811c3-c06e-4feb-a246-90a0cdfae81c · outbound

This paper cites The economic potential of generative AI: The next productivity frontier.

Leveraging LLM Inconsistency to Boost Pass@k Performance The economic potential of generative AI: The next productivity frontier

Reference 1

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.940072Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 14a3b913-0044-45f4-8b0a-6f8f49158bcd · outbound

This paper cites The language of prompting: What linguistic properties make a prompt successful?, 2023.

Leveraging LLM Inconsistency to Boost Pass@k Performance The language of prompting: What linguistic properties make a prompt successful?, 2023

Reference 2

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.924468Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 84dff12b-de9d-4e32-b417-cdac8815abee · outbound

This paper cites Robustness of learning from task instructions, 2022.

Leveraging LLM Inconsistency to Boost Pass@k Performance Robustness of learning from task instructions, 2022

Reference 3

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.910041Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 049659ac-5db1-4a38-b987-a7c69bbbc802 · outbound

This paper cites Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions.

Leveraging LLM Inconsistency to Boost Pass@k Performance Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:12.974774Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bc25770c-8471-4f25-a152-9430e640b13e · outbound

This paper cites When benchmarks are targets: Revealing the sensitivity of large language model leaderboards.

Leveraging LLM Inconsistency to Boost Pass@k Performance When benchmarks are targets: Revealing the sensitivity of large language model leaderboards

Reference 5

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.894304Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 4045c549-ac84-4aca-84fe-b4e25b88e8cc · outbound

This paper cites PertEval: Unveiling real knowledge capacity of LLMs with knowledge-invariant perturbations.

Leveraging LLM Inconsistency to Boost Pass@k Performance PertEval: Unveiling real knowledge capacity of LLMs with knowledge-invariant perturbations

Reference 6

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.878317Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation b544a02a-374a-432a-8078-8752f5855fc1 · outbound

This paper cites ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs.

Leveraging LLM Inconsistency to Boost Pass@k Performance ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:12.997523Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ae268e7d-e0cf-4f6f-9657-308e2f99f518 · outbound

This paper cites Mind your format: Towards consistent evaluation of in-context learning improvements, 2024.

Leveraging LLM Inconsistency to Boost Pass@k Performance Mind your format: Towards consistent evaluation of in-context learning improvements, 2024

Reference 8

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.863787Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 20ae5528-f764-4874-8795-a67ee1de55a7 · outbound

This paper cites State of what art? A call for multi-prompt LLM evaluation.

Leveraging LLM Inconsistency to Boost Pass@k Performance State of what art? A call for multi-prompt LLM evaluation

Reference 9

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.849170Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 75fb61e9-5df3-42e7-bbdf-f77d73b08202 · outbound

This paper cites Unnatural instructions: Tuning language models with (almost) no human labor, 2022.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unnatural instructions: Tuning language models with (almost) no human labor, 2022

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.834279Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 21ace7bd-d718-4556-80b4-7b331f75d834 · outbound

This paper cites Evaluating the Zero-shot Robustness of Instruction-tuned Language Models.

Leveraging LLM Inconsistency to Boost Pass@k Performance Evaluating the Zero-shot Robustness of Instruction-tuned Language Models

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.021272Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6114d2c4-7047-4f56-b94b-82d5f168f982 · outbound

This paper cites Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting.

Leveraging LLM Inconsistency to Boost Pass@k Performance Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.026850Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 26236707-aa42-4280-96b6-a4e75b7814e8 · outbound

This paper cites Evaluating Large Language Models Trained on Code.

Leveraging LLM Inconsistency to Boost Pass@k Performance Evaluating Large Language Models Trained on Code

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.032426Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation dfa58b10-3b05-4093-b2e4-078cb97018bb · outbound

This paper cites Spoc: Search-based pseudocode to code.

Leveraging LLM Inconsistency to Boost Pass@k Performance Spoc: Search-based pseudocode to code

Reference 14

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.819052Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 40548b60-6a9a-4541-a324-1e52a2e3f06d · outbound

This paper cites AI-powered fuzzing: Breaking the bug hunting barrier, 2023.

Leveraging LLM Inconsistency to Boost Pass@k Performance AI-powered fuzzing: Breaking the bug hunting barrier, 2023

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.787428Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation a79e3131-e466-4cec-bb40-475eaf9586ed · outbound

This paper cites OpenAI o1 System Card.

Leveraging LLM Inconsistency to Boost Pass@k Performance OpenAI o1 System Card

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.055766Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 055843a1-3ca4-4375-98ab-2084d87348d7 · outbound

This paper cites OpenAI o3-mini system card, 2025.

Leveraging LLM Inconsistency to Boost Pass@k Performance OpenAI o3-mini system card, 2025

Reference 17

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.772214Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation b7b90270-1a70-4c12-b448-2cfb644155c7 · outbound

This paper cites Claude 3.7 Sonnet system card, 2025.

Leveraging LLM Inconsistency to Boost Pass@k Performance Claude 3.7 Sonnet system card, 2025

Reference 18

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.757034Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 64acb600-91b5-4841-b4e3-4981a1682fab · outbound

This paper cites Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity.

Leveraging LLM Inconsistency to Boost Pass@k Performance Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.071631Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1025282a-beb9-40cd-904b-b52a063da18c · outbound

This paper cites POSIX: A Prompt Sensitivity Index For Large Language Models.

Leveraging LLM Inconsistency to Boost Pass@k Performance POSIX: A Prompt Sensitivity Index For Large Language Models

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.077068Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 054d1558-6a52-4af8-9aa3-f9d6a35e9e8f · outbound

This paper cites DOVE: A large-scale multi-dimensional predictions dataset towards meaningful LLM evaluation, 2025.

Leveraging LLM Inconsistency to Boost Pass@k Performance DOVE: A large-scale multi-dimensional predictions dataset towards meaningful LLM evaluation, 2025

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.742021Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 6d85e5b4-b88d-4886-af06-4796969e0924 · outbound

This paper cites Demystifying Prompts in Language Models via Perplexity Estimation.

Leveraging LLM Inconsistency to Boost Pass@k Performance Demystifying Prompts in Language Models via Perplexity Estimation

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.086775Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7db39a4e-a403-4208-9533-4ba61f1bc2c1 · outbound

This paper cites Measuring Coding Challenge Competence With APPS.

Leveraging LLM Inconsistency to Boost Pass@k Performance Measuring Coding Challenge Competence With APPS

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.091733Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 917100e3-ff85-415f-831c-f0d98fd59828 · outbound

This paper cites NYU CTF bench: A scalable open-source benchmark dataset for evaluating LLMs in offensive security.

Leveraging LLM Inconsistency to Boost Pass@k Performance NYU CTF bench: A scalable open-source benchmark dataset for evaluating LLMs in offensive security

Reference 24

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.726480Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation ebdbc089-1e15-4316-b6f6-18b6038ba51b · outbound

This paper cites OpenAI o3 and o4-mini system card, 2025.

Leveraging LLM Inconsistency to Boost Pass@k Performance OpenAI o3 and o4-mini system card, 2025

Reference 25

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.711876Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation c80dba00-874f-4d64-8dbd-0988ac42796d · outbound

This paper cites A Framework for Evaluating Emerging Cyberattack Capabilities of AI.

Leveraging LLM Inconsistency to Boost Pass@k Performance A Framework for Evaluating Emerging Cyberattack Capabilities of AI

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:13.107210Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9e313fbb-6ff4-4599-9e99-b8655b4cb042 · outbound

This paper cites Amazon Bedrock.

Leveraging LLM Inconsistency to Boost Pass@k Performance Amazon Bedrock

Reference 27

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.697516Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 173b315b-979d-4164-814d-1bb3fe68309a · outbound

This paper cites Openai api.

Leveraging LLM Inconsistency to Boost Pass@k Performance Openai api

Reference 28

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.683092Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 44a75d7f-4cc0-4f3a-be6a-3103d3e648ac · outbound

This paper cites Azure OpenAI service.

Leveraging LLM Inconsistency to Boost Pass@k Performance Azure OpenAI service

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.669725Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 8002fa64-242f-4fc9-97d0-25b4c2c7c079 · outbound

This paper cites pickle — python object serialization, 2025.

Leveraging LLM Inconsistency to Boost Pass@k Performance pickle — python object serialization, 2025

Reference 30

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.655720Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation a7716a26-4cf5-424e-9f0c-5ae892af3ffb · outbound

This paper cites t2.large.

Leveraging LLM Inconsistency to Boost Pass@k Performance t2.large

Reference 31

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.640559Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation bc4a04d9-a225-4bc6-81e7-18979a6a2e34 · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 34

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.626295Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 44338ff1-5d1d-400f-9f68-6eb96beb422d · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 35

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.612306Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 9617a151-5a42-409d-b4fc-b5579a5c0c2d · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 36

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.597551Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 2efb0b7a-70ca-4c43-b77e-4d339918589f · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 37

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.582236Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 5d2254f8-d1c3-43cf-8c45-a707ecfb012a · outbound

This paper cites In this appendix, we provide the instructions given to these experts.

Leveraging LLM Inconsistency to Boost Pass@k Performance In this appendix, we provide the instructions given to these experts

Reference 38

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.567896Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 53fb84d6-9d9a-4fc7-9487-349c5dacae30 · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 39

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.553625Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation adf67d3b-3de4-404e-95c2-efa27b169ac6 · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 40

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.539235Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 5b316919-b601-4fb0-99a6-3bbb72b51c03 · outbound

This paper cites Translating.

Leveraging LLM Inconsistency to Boost Pass@k Performance Translating

Reference 41

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.524147Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation d36779ed-a6a7-42cc-8f58-efcc23fd7338 · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 42

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.507631Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation c4ba2d4d-cb9d-408f-9868-154dd03580e7 · outbound

This paper cites ) Let me solve this step by step.

Leveraging LLM Inconsistency to Boost Pass@k Performance ) Let me solve this step by step

Reference 43

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.490929Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 0192c96a-0268-4e19-ba52-77e741299a75 · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 44

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.475571Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 1c981aa7-b035-4c2b-b67b-37abf5c2020c · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 45

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.460555Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation dd5bee7b-a92a-4eb1-a301-f0289d7caba0 · outbound

This paper cites mph” to “leaves per day,.

Leveraging LLM Inconsistency to Boost Pass@k Performance mph” to “leaves per day,

Reference 46

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.445316Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 3493161a-5832-472a-9359-769bbdbb89b4 · outbound

This paper cites Sample the results multiple times to obtain an average success rate.

Leveraging LLM Inconsistency to Boost Pass@k Performance Sample the results multiple times to obtain an average success rate

Reference 47

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T20:30:13.429803Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 6a455ac9-2075-46a7-bf10-9e1f36cfe491 · outbound

This paper cites an unresolved cited work.

Leveraging LLM Inconsistency to Boost Pass@k Performance Unresolved cited work

Reference 2019

Resolution: unresolved; raw_fallback, observed 2026-08-15T20:30:13.803622Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 739490a0-e31e-487f-a4dc-8c1d5677c20a · outbound

This paper cites PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations.

Leveraging LLM Inconsistency to Boost Pass@k Performance PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-15T20:30:12.991801Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation dc005f4f-05d3-4139-a303-5adb886c2fb3 · inbound

Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation cites this paper.

Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation Leveraging LLM Inconsistency to Boost Pass@k Performance

Reference 41

Resolution: verified exact; arxiv_id, observed 2026-05-18T10:06:13.819298Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.