Pith. sign in

REVIEW 3 major objections 4 minor 30 references

This paper proposes Proof-Carrying Numbers (PCN), a protocol in which every number an LLM displays is either mechanically verified against a structured claim under a declared policy or explicitly shown as unverified.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 22:50 UTC pith:QB75VQXD

load-bearing objection PCN is a clean, honest protocol for renderer-side numeric verification, but its headline guarantee only holds conditional on the LLM citing the right claim—something the paper acknowledges obliquely and never tests. the 3 major comments →

arxiv 2509.06902 v1 pith:QB75VQXD submitted 2025-09-08 cs.CL cs.CRcs.DBcs.LG

Proof-Carrying Numbers (PCN): A Protocol for Trustworthy Numeric Answers from LLMs via Claim Verification

Aivin V. Solatorio This is my paper
classification cs.CL cs.CRcs.DBcs.LG
keywords proof-carrying numbersnumeric hallucinationclaim verificationfail-closed protocolLLM trustprovenanceverification policyrenderer-side verification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that numeric hallucination in LLMs is best fixed not by making the model more truthful but by controlling what the interface is permitted to mark as verified. It proposes Proof-Carrying Numbers (PCN), a presentation-layer protocol in which every numeric span is emitted either as a claim-bound token tied to a structured claim or as a bare number. A verifier running in the renderer applies a declared policy—exact match, rounding, alias equivalence, or tolerance with qualifiers—and marks a number as verified only when it deterministically matches a claim. If this works as described, users can treat a verified badge as proof of correspondence to an authoritative source, while the absence of a mark honestly signals uncertainty.

Core claim

PCN frames numeric hallucination as a presentation-layer problem: even when correct values are retrievable, the model may reproduce them incorrectly and the UI has no way to tell. The protocol requires values to appear as <claim id="CID" policy="P">VAL</claim> tokens, binds each token to a claim in the resolved claim set, and defines a verification relation that holds when the normalized value matches under the active policy. Verified status is computed by the renderer, not the model, so text like "verified" or checkmark symbols cannot spoof it; numbers without a valid claim reference are always unverified. The paper proves soundness, completeness under honest tokens, fail-closed behavior, m

What carries the argument

The central object is the claim-bound token: a numeric span wrapped as <claim id="CID" policy="P">VAL</claim>. It links a surface number to a structured claim c = ⟨claim id, indicator, entity, time, v*, u, m⟩. The verifier's relation R(t,c; Π) then decides acceptance using policy modes (exact, rounded, alias, tolerance with qualifiers). Because the renderer computes R, verification cannot be influenced by the model's text, which is what makes the fail-closed contract hold.

Load-bearing premise

The model must attach each number to the claim that actually answers the user's question; the verifier only checks that the number matches the cited claim's value, not that the claim is the right one.

What would settle it

Run an end-to-end PCN system where the model answers a question about one indicator but binds the number to a claim for a different indicator with the same value. If the renderer displays the Verified badge, the protocol's guarantee as stated—that verified means honestly sourced—fails. The same test with a claim whose value matches only after an overly permissive tolerance policy would also show where the policy boundary must be tightened.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A verified badge becomes a binding statement: the displayed number equals, rounds to, aliases, or tolerates to a value in a claim the system retrieved.
  • Any number without a claim tag is displayed unverified by default, so fabricated values cannot appear as fact even if the surrounding text looks fluent.
  • Policy presets let applications trade strictness for coverage predictably: tightening a policy only removes verified marks, never adds false ones.
  • Verification adds negligible latency—O(n) over numeric spans—so it can ride on existing RAG and chat pipelines.
  • With signatures or Merkle proofs, the same contract can certify that the claim itself has not been tampered with.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The load-bearing step is the model's choice of which claim ID to attach; the verifier checks numeric consistency only. A model could bind a plausible number to an irrelevant but numerically matching claim, and the renderer would mark it Verified. A robust design should verify claim relevance to the query, not just value equality.
  • The protocol could be extended to derived numbers (ratios, aggregates) by verifying deterministic functions over atomic claims; the paper lists this as future work, but the same renderer-side contract would then need to check the derivation, not just the final span.
  • User trust depends on whether people understand that an unmarked number is merely unverified, not necessarily wrong; controlled studies of badge interpretation would tell whether the interface actually changes behavior.
  • Adoption incentives may be the real bottleneck: data custodians become trust anchors and may hesitate to publish canonical claims without governance frameworks; this is a deployment condition, not a protocol flaw.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Proof-Carrying Numbers (PCN), a presentation-layer protocol intended to prevent numeric hallucination in LLM outputs. PCN requires numeric spans to be emitted as claim-bound tokens referencing structured claims; a renderer-side verifier checks each token against the referenced claim under a configurable policy (exact, rounding, alias, tolerance with qualifiers) and labels it Verified or Unverified. The authors formalize the protocol, state soundness, completeness, fail-closed, monotonicity, and renderer-robustness properties, and provide a TypeScript implementation sketch in the appendix. The central design idea is that verification status is computed by the renderer, not by the model, so fabricated numbers cannot display a verified badge unless they match a claim in the available claim set.

Significance. If the practical assumptions are met, PCN would provide a clean, testable contract for numeric claims in LLM interfaces: verified numbers are always traceable to a structured claim, and unverified numbers are visibly unmarked. The separation of verification from generation is a useful contribution and the policy layer is a sensible way to express application-specific tolerance. The appendix gives a concrete, executable implementation sketch, which strengthens reproducibility. However, the formal guarantees are largely definitional, and the protocol's practical value depends on two unverified assumptions: that LLMs can reliably emit claim-bound tokens with correct claim IDs, and that numeric consistency with some claim implies relevance to the user's query. The paper's advertised user contract ('trustworthy numeric answers') is stronger than the formal problem statement, which only ensures existence of some matching claim in C.

major comments (3)
  1. [§3.6, §5.1, §4.4] The central soundness guarantee is weaker than the paper's advertised contract. The acceptance function A labels a token as Verified iff there exists some c∈C with R(t,c;Π)=1; the verifier does not access the user query q and cannot check that c is the claim that answers q. Consequently, a numerically incorrect answer can receive a Verified badge if its value coincides with any distractor claim in C (e.g., a different country or year). Section 6.1 correctly limits the claim to 'verifiably consistent with a claim under a policy Π', but the abstract and §4.4's user contract promise 'trustworthy numeric answers' and treat the badge as a fidelity guarantee. Section 6.3 mentions 'Identifier abuse' only indirectly and does not flag that claim-ID semantic correctness is a load-bearing unverified assumption. The paper should either consistently restate the guarantee as 'consistency with some cla
  2. [§6.4, §6.6] The protocol's practical value rests on the LLM's ability to emit claim-bound tokens with correct claim IDs and values at high recall. The paper states in §6.4 that 'Early experiments suggest that prompting or light fine-tuning enables models to emit claim tags with reasonable recall' and in §6.6 that effectiveness 'hinges on LLM cooperation', but no experimental data, dataset, or measurement are provided. Without any evaluation of token-emission compliance and claim-ID correctness, the feasibility claims in the abstract and §4.2 are unsupported. This is a major gap for a cs.CL paper; even a small-scale study measuring compliance rates and error types would substantially strengthen the contribution.
  3. [§5.1–§5.4] Theorems 5.1–5.4 are definitional rather than substantive. Because A is defined to label a token Verified iff R(t,c;Π)=1 for some c, each proof sketch simply unpacks the definition of A. These results do not establish that verified numbers are correct with respect to the query or that the underlying claims are trustworthy; they only restate the protocol's specification. The section would be more honest if reframed as a formal specification with invariants, with the substantive arguments focused on policy monotonicity, renderer robustness, and the practical properties that require reasoning beyond definitions (e.g., the relationship between token syntax and the renderer's parsing behavior).
minor comments (4)
  1. [§5.1, §6.2] Lemma 5.4 is called 'Theorem 5.4' in §6.2. Please make the numbering and theorem/lemma labels consistent.
  2. [§3.4, §A.1] The normalization step is under-specified ('normalized into the claim's unit'), and the sample implementation strips only spaces and commas, not '%' or other units. This makes the exact-policy example in §A.1 depend on a formatting coincidence. Please specify normalization precisely and align the code with the formal definition.
  3. [§3.4] The definitions of round_d, round0, and the alias set S are informal. For example, does rounding operate on the displayed string or the underlying float, and how are negative values handled? A compact formal definition would remove ambiguity.
  4. [Figures 2 and 3] The figures are referenced but not described in the main text; their captions should state what behavior they illustrate (e.g., a verified badge vs. a pending/warning mark) so the reader can interpret them without guessing.

Circularity Check

5 steps flagged

PCN's formal guarantees (Soundness, Completeness, Fail-Closed, Monotonicity, Renderer Robustness) are restatements of the definition of the acceptance function A and the verification relation R, so the 'Verified' label is by construction the label assigned when R=1.

specific steps
  1. self definitional [Section 5.1, Theorem 5.1 (Soundness) proof sketch]
    "By construction, the verifier only assigns Verified if it finds such a claim. Hence no fabricated value can be marked as Verified."

    The acceptance function A is defined in Section 5's preamble to label each span as Verified or Unverified, and the verifier assigns Verified exactly when R(t,c;Π)=1 for some c∈C (Sections 3.4 and 4.2). Theorem 5.1 therefore restates the definition of 'Verified'; it adds no independent constraint. The claim that PCN 'guarantees fidelity' in this sense is true solely because fidelity is defined as matching some claim.

  2. self definitional [Section 5.1, Theorem 5.2 (Completeness under honest tokens) proof sketch]
    "Determinism of the verifier ensures all policy-compliant tokens are accepted."

    Policy-compliant tokens are, by definition (Section 3.4), tokens for which R(t,c;Π)=1; the verifier is defined to accept exactly those. Thus completeness is not a derived property but the definition of the verifier's behavior. It also assumes the generator already emits the correct claim ID, which is the unverified semantic link.

  3. self definitional [Section 5.1, Theorem 5.3 (Fail-Closed) proof sketch]
    "The acceptance function defaults to Unverified unless an explicit match is found."

    The fail-closed property is written into the definition of A: any span not matching R under Π is labeled Unverified. The theorem restates the default branch of the acceptance function. It does not establish that unmarked numbers are wrong, only that they are unverified by definition.

  4. self definitional [Section 5.1, Lemma 5.4 (Monotonicity under policy refinement) proof sketch]
    "Tightening policies reduces coverage but never introduces false positives."

    If Π1 is defined as stricter than Π2 (Π1 ⪯ Π2), then R(t,c;Π1)=1 implies R(t,c;Π2)=1 by the mode-inclusion definition of Π (Section 3.4). The subset inclusion of Verified sets is a direct consequence of the ordering on policies, not a separate finding.

  5. self definitional [Section 5.2, Theorem 5.5 (Renderer robustness) proof sketch]
    "Verified status is derived solely from R(t,c; Π). Spoofed tokens are ignored by the parser and remain Unverified."

    The theorem's antecedent ('if verification status is computed by the renderer rather than text tokens') is exactly the PCN architecture; the conclusion follows because the renderer's label is defined as the output of R. The anti-spoofing guarantee is a restatement of the design decision that text cannot set the status.

full rationale

The paper's formal 'correctness guarantees' are all direct restatements of its own definitions. The acceptance function A is defined to label a numeric span Verified exactly when the verification relation R(t,c;Π)=1 holds, and Unverified otherwise; Theorems 5.1–5.3 and Lemma 5.4 unpack that definition. Similarly, renderer robustness (Theorem 5.5) is the definition of placing verification in the renderer. There are no fitted parameters, no empirical predictions, and no load-bearing self-citations, so the usual circularity patterns do not apply. Instead, the circularity is self-definitional: the central guarantee—that a Verified number is consistent with some claim under Π—is true because 'Verified' was defined as that consistency. The paper is careful to scope this as 'correspondence to a chosen source, not ultimate truth' (Section 7), and it acknowledges that claim-ID semantic correctness is outside the formal contract (Sections 6.1, 6.3). But the abstract and concluding 'trust is earned only by proof' language invites the reader to treat the definitional property as an empirical or cryptographic guarantee that hallucinated numbers cannot be marked Verified. In fact, a numerically consistent but semantically irrelevant claim will pass verification, as the running example's claim-bound token mechanism shows. That limitation is real but is not itself circularity; the circularity lies in presenting the protocol's own definitions as derived theorems. Score 8 reflects that the main formal results are forced by definition, while leaving room for the genuinely non-circular implementation details (claim aggregation, hash-table verification, optional cryptographic commitments).

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 1 invented entities

The protocol itself avoids fitted numerical quantities; all numeric verification is deterministic. The burden is carried by domain assumptions about retrieval quality, LLM tag emission and claim-ID assignment, and renderer trust. No physical or explanatory entities are introduced, only a data format.

free parameters (1)
  • Policy parameters (rounding precision d, tolerance delta/rho, alias set S) = user-selected per application (e.g., round1, int, about 0.1)
    These parameters are declared by the application and determine which numbers count as Verified. They are not derived from data and are arbitrary choices governing the strictness/coverage trade-off.
axioms (5)
  • domain assumption A retriever resolves the user query into a finite claim set C that contains the authoritative values.
    Section 3.2 introduces C; if retrieval returns wrong or incomplete claims, PCN verifies against those claims and can still display numbers that are wrong for the query.
  • domain assumption The LLM reliably emits claim-bound tokens with the correct claim ID and the intended numeric value inside the tag.
    Section 3.3 and Section 6.3 list poor tagging recall as a known limitation; if the model omits tags, numbers stay unverified, and if it assigns a wrong but matching claim ID, the verifier will still accept them.
  • domain assumption The assignment of a claim ID to a numeric span is semantically meaningful with respect to the user's question.
    Section 3.5's running example and Section 6.1's scope statement make clear that PCN only checks consistency with the referenced claim, not whether that claim answers the query.
  • domain assumption The renderer executes the verifier faithfully and does not render attacker-controlled HTML or Unicode as a verification mark.
    Theorem 5.5 assumes spoofed tokens are ignored by the parser; the appendix regex parser does not fully specify HTML sanitization, so this is an assumption about the implementation.
  • standard math EUF-CMA signature security and hash collision resistance for the cryptographic extension.
    Theorem 5.7 depends on these standard assumptions.
invented entities (1)
  • Claim-bound token syntax (<claim id="CID" policy="P">VAL</claim>) no independent evidence
    purpose: Binds each LLM-generated numeric span to a structured claim so the verifier can mechanically check it.
    This is the core new artifact of the protocol. No deployed implementation or benchmark demonstrates that it works at scale; evidence is limited to an illustrative TypeScript snippet.

pith-pipeline@v1.3.0-alltime-deepseek · 11121 in / 14368 out tokens · 140365 ms · 2026-08-04T22:50:57.886212+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Proof-Carrying Numbers (PCN): A Protocol for Trustworthy Numeric Answers from LLMs via Claim Verification." pith.science (2026). https://pith.science/paper/QB75VQXD

@misc{pith2026250906902,
  author       = {Pith},
  title        = {Pith review of: Proof-Carrying Numbers (PCN): A Protocol for Trustworthy Numeric Answers from LLMs via Claim Verification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QB75VQXD}},
  note         = {Machine review of arXiv:2509.06902}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) as stochastic systems may generate numbers that deviate from available data, a failure known as \emph{numeric hallucination}. Existing safeguards -- retrieval-augmented generation, citations, and uncertainty estimation -- improve transparency but cannot guarantee fidelity: fabricated or misquoted values may still be displayed as if correct. We propose \textbf{Proof-Carrying Numbers (PCN)}, a presentation-layer protocol that enforces numeric fidelity through mechanical verification. Under PCN, numeric spans are emitted as \emph{claim-bound tokens} tied to structured claims, and a verifier checks each token under a declared policy (e.g., exact equality, rounding, aliases, or tolerance with qualifiers). Crucially, PCN places verification in the \emph{renderer}, not the model: only claim-checked numbers are marked as verified, and all others default to unverified. This separation prevents spoofing and guarantees fail-closed behavior. We formalize PCN and prove soundness, completeness under honest tokens, fail-closed behavior, and monotonicity under policy refinement. PCN is lightweight and model-agnostic, integrates seamlessly into existing applications, and can be extended with cryptographic commitments. By enforcing verification as a mandatory step before display, PCN establishes a simple contract for numerically sensitive settings: \emph{trust is earned only by proof}, while the absence of a mark communicates uncertainty.

Figures

Figures reproduced from arXiv: 2509.06902 by Aivin V. Solatorio.

Figure 1
Figure 1. Figure 1: PCN-compliant architecture with LLM-initiated retrieval. The LLM first parses [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Implementation of the Proof-Carrying Numbers (PCN) protocol in an LLM chat [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the Proof-Carrying Numbers (PCN) protocol in an LLM chat [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 10 canonical work pages

  1. [1]

    FEVEROUS : Fact Extraction and VERification Over Unstructured and Structured information

    Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. FEVEROUS : Fact Extraction and VERification Over Unstructured and Structured information. June 2021. URL https://openreview.net/forum?id=h-flVCIlstW

    work page 2021

  2. [2]

    Introducing the Model Context Protocol

    Anthropic. Introducing the Model Context Protocol . URL https://www.anthropic.com/news/model-context-protocol

    work page

  3. [3]

    LLMs Will Always Hallucinate , and We Need to Live With This , September 2024

    Sourav Banerjee, Ayushi Agarwal, and Saloni Singla. LLMs Will Always Hallucinate , and We Need to Live With This , September 2024. URL http://arxiv.org/abs/2409.05746. arXiv:2409.05746 [stat]

    Pith/arXiv arXiv 2024

  4. [4]

    TabFact : A Large -scale Dataset for Table -based Fact Verification , June 2020

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. TabFact : A Large -scale Dataset for Table -based Fact Verification , June 2020. URL http://arxiv.org/abs/1909.02164. arXiv:1909.02164 [cs]

    Pith/arXiv arXiv 2020

  5. [5]

    Detecting hallucinations in large language models using semantic entropy

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630 0 (8017): 0 625--630, June 2024. ISSN 1476-4687. doi:10.1038/s41586-024-07421-0. URL https://www.nature.com/articles/s41586-024-07421-0. Publisher: Nature Publishing Group

    doi:10.1038/s41586-024-07421-0 2024

  6. [6]

    Grammar- Constrained Decoding for Structured NLP Tasks without Finetuning

    Saibo Geng, Martin Josifoski, Maxime Peyrard, and Robert West. Grammar- Constrained Decoding for Structured NLP Tasks without Finetuning . In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pp.\ 10932--10952, Singapore, December 2023. Association for Computational L...

    doi:10.18653/v1/2023.emnlp-main.674 2023

  7. [7]

    Hakim, Jeffery L

    Joe B. Hakim, Jeffery L. Painter, Darmendra Ramcharran, Vijay Kara, Greg Powell, Paulina Sobczak, Chiho Sato, Andrew Bate, and Andrew Beam. The need for guardrails with large language models in pharmacovigilance and other medical safety critical settings. Scientific Reports, 15 0 (1): 0 27886, July 2025. ISSN 2045-2322. doi:10.1038/s41598-025-09138-0. URL...

    work page doi:10.1038/s41598-025-09138-0 2025

  8. [8]

    Towards Verifiable Text Generation with Symbolic References

    Lucas Torroba Hennigen, Zejiang Shen, Aniruddha Nrusimha, Bernhard Gapp, David Sontag, and Yoon Kim. Towards Verifiable Text Generation with Symbolic References . August 2024. URL https://openreview.net/forum?id=fib9qidCpY

    work page 2024

  9. [9]

    Survey of Hallucination in Natural Language Generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of Hallucination in Natural Language Generation . ACM Comput. Surv., 55 0 (12): 0 248:1--248:38, March 2023. ISSN 0360-0300. doi:10.1145/3571730. URL https://dl.acm.org/doi/10.1145/3571730

    doi:10.1145/3571730 2023

  10. [10]

    Language Models ( Mostly ) Know What They Know , November 2022

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec,...

    Pith/arXiv arXiv 2022

  11. [11]

    Vempala, and Edwin Zhang

    Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. Why language models hallucinate, September 2025. URL https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf

    work page 2025

  12. [12]

    Deficiency of Large Language Models in Finance : An Empirical Examination of Hallucination , November 2023

    Haoqiang Kang and Xiao-Yang Liu. Deficiency of Large Language Models in Finance : An Empirical Examination of Hallucination , November 2023. URL http://arxiv.org/abs/2311.15548. arXiv:2311.15548 [cs]

    Pith/arXiv arXiv 2023

  13. [13]

    Medical Hallucination in Foundation Models and Their Impact on Healthcare , March 2025

    Yubin Kim, Hyewon Jeong, Shan Chen, Shuyue Stella Li, Mingyu Lu, Kumail Alhamoud, Jimin Mun, Cristina Grau, Minseok Jung, Rodrigo Gameiro, Lizhou Fan, Eugene Park, Tristan Lin, Joonsik Yoon, Wonjin Yoon, Maarten Sap, Yulia Tsvetkov, Paul Liang, Xuhai Xu, Xin Liu, Daniel McDuff, Hyeonhoon Lee, Hae Won Park, Samir Tulebaev, and Cynthia Breazeal. Medical Hal...

    work page doi:10.1101/2025.02.28.25323115v1 2025

  14. [14]

    Retrieval-augmented generation for knowledge-intensive NLP tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS '...

    work page 2020

  15. [15]

    Yifei Li, Xiang Yue, Zeyi Liao, and Huan Sun. AttributionBench : How Hard is Automatic Attribution Evaluation ? In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics : ACL 2024 , pp.\ 14919--14935, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi:10.18653/v1/2024.fi...

    doi:10.18653/v1/2024.findings-acl.886 2024

  16. [16]

    SelfCheckGPT : Zero - Resource Black - Box Hallucination Detection for Generative Large Language Models

    Potsawee Manakul, Adian Liusie, and Mark Gales. SelfCheckGPT : Zero - Resource Black - Box Hallucination Detection for Generative Large Language Models . In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pp.\ 9004--9017, Singapore, December 2023. Association for Co...

    doi:10.18653/v1/2023.emnlp-main.557 2023

  17. [17]

    George C. Necula. Proof-carrying code. In Proceedings of the 24th ACM SIGPLAN - SIGACT symposium on Principles of programming languages , ACM Conferences , pp.\ 106--119. January 1997. ISBN 978-0-89791-853-4. doi:10.1145/263699.263712. URL https://dl.acm.org/doi/10.1145/263699.263712

    arXiv 1997

  18. [18]

    Attribution, Citation , and Quotation : A Survey of Evidence -based Text Generation with Large Language Models , August 2025

    Tobias Schreieder, Tim Schopf, and Michael Färber. Attribution, Citation , and Quotation : A Survey of Evidence -based Text Generation with Large Language Models , August 2025. URL http://arxiv.org/abs/2508.15396. arXiv:2508.15396 [cs] version: 1

    Pith/arXiv arXiv 2025

  19. [19]

    World Bank Open Data - World Development Indicators ( WDI ), 2025

    The World Bank . World Bank Open Data - World Development Indicators ( WDI ), 2025. URL https://data.worldbank.org

    work page 2025

  20. [20]

    FEVER : a Large -scale Dataset for Fact Extraction and VERification

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER : a Large -scale Dataset for Fact Extraction and VERification . In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies , Volume 1 (...

    doi:10.18653/v1/n18-1074 2018

  21. [21]

    Verifiable Credential Data Integrity 1.0

    W3C. Verifiable Credential Data Integrity 1.0. URL https://www.w3.org/TR/vc-data-integrity/

    work page

  22. [22]

    SciFact - Open : Towards open-domain scientific claim verification

    David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, and Hannaneh Hajishirzi. SciFact - Open : Towards open-domain scientific claim verification. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics : EMNLP 2022 , pp.\ 4719--4734, Abu Dhabi, United Arab Emirates, Decemb...

    doi:10.18653/v1/2022.findings-emnlp.347 2022

  23. [23]

    An automated framework for assessing how well LLMs cite relevant medical references

    Kevin Wu, Eric Wu, Kevin Wei, Angela Zhang, Allison Casasola, Teresa Nguyen, Sith Riantawan, Patricia Shi, Daniel Ho, and James Zou. An automated framework for assessing how well LLMs cite relevant medical references. Nature Communications, 16 0 (1): 0 3615, April 2025 a . ISSN 2041-1723. doi:10.1038/s41467-025-58551-6. URL https://www.nature.com/articles...

    doi:10.1038/s41467-025-58551-6 2025

  24. [24]

    ClashEval : Quantifying the tug-of-war between an LLM 's internal prior and external evidence, February 2025 b

    Kevin Wu, Eric Wu, and James Zou. ClashEval : Quantifying the tug-of-war between an LLM 's internal prior and external evidence, February 2025 b . URL http://arxiv.org/abs/2404.10198. arXiv:2404.10198 [cs]

    Pith/arXiv arXiv 2025

  25. [25]

    Hallucination is Inevitable : An Innate Limitation of Large Language Models , February 2025

    Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucination is Inevitable : An Innate Limitation of Large Language Models , February 2025. URL http://arxiv.org/abs/2401.11817. arXiv:2401.11817 [cs]

    Pith/arXiv arXiv 2025

  26. [26]

    LongCite : Enabling LLMs to Generate Fine -grained Citations in Long - Context QA

    Jiajie Zhang, Yushi Bai, Xin Lv, Wanjun Gu, Danqing Liu, Minhao Zou, Shulin Cao, Lei Hou, Yuxiao Dong, Ling Feng, and Juanzi Li. LongCite : Enabling LLMs to Generate Fine -grained Citations in Long - Context QA . In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics...

    doi:10.18653/v1/2025 2025

  27. [27]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  28. [28]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  29. [29]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  30. [30]

    CID" policy=

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page