Pith. sign in

REVIEW 3 major objections 6 minor 4 cited by

Knowledge Editing through Chain-of-Thought

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EditCoT rewrites a model's chain-of-thought to update its knowledge without retraining.

desk verdict A genuinely useful CoT-editing framework with honest ablations, undermined by an overclaimed SOTA headline that its own tables refute. read the letter →

arxiv 2412.17727 v2 pith:LIJTOWFF submitted 2024-12-23 cs.CL

Changyue Wang , Weihang Su , Qingyao Ai , Yichen Tang , Yiqun Liu This is my paper · ORCID
classification cs.CL
keywords knowledgeeditingin-contextchain-of-thoughtiterativereasoningrefinementmulti-hopquestionansweringlegalupdateblack-boxLLMsconflictdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EditCoT is a knowledge-editing framework that claims to update an LLM's factual knowledge by rewriting the model's chain of thought rather than retraining or changing weights. The method has the model answer a question, generate a step-by-step reasoning chain, retrieve updated facts, detect which steps conflict with those facts, and iteratively hand conflicting chains to a separately trained editor that produces a corrected chain. The paper reports that this one pipeline, with no task-specific retuning, reaches the best or second-best accuracy on multi-hop triple-based QA, free-form scientific and arithmetic edits, and Chinese legal statute updates, in both English and Chinese. A sympathetic reader would care because it points toward a way to keep deployed models current without expensive retraining, while preserving the model's original abilities.

What carries the argument

The load-bearing mechanism is the iterative edit loop built around a small, task-agnostic CoT editor. The editor is trained once, on automatically generated examples derived from HotpotQA questions, to take a question, an old chain, and new knowledge and output a revised chain; no retraining is needed when the knowledge changes. In inference it is driven by a three-way conflict classifier (conflict, support, unrelated) that decides whether the current reasoning step disagrees with retrieved updated facts, and by a retriever that maps old facts to their replacements or finds post-edit knowledge directly.

What would settle it

Run EditCoT on MQuAKE-CF-3k with a conflict detector replaced by an oracle that knows ground-truth conflicts, and compare with the model-based detector; if accuracy does not improve, conflict detection is not the mechanism the paper claims. The mirror test is to inject a fixed false conflict on unrelated retrieved facts and count how often the final answer flips; the paper's Table 9 already shows one such flip, so a dataset where this happens frequently would refute the stability claim.

Watch

Extended reading notes

Core claim

The central discovery claimed is that knowledge conflicts can be resolved at the level of the reasoning trace. EditCoT treats a chain-of-thought as an editable object: the unedited LLM generates the chain and classifies each retrieved fact as support, conflict, or unrelated; on a conflict, a trained CoT editor rewrites the chain; the loop repeats until no conflict remains; and the edited chain then drives the final answer. The paper reports state-of-the-art or near-state-of-the-art accuracy across MQuAKE-CF-3k, DUNE, and LeKUBE on Llama-3-8B-Instruct and Qwen2.5-14B-Instruct, plus competitive results on GPT-4o using an 8B editor as a proxy, and shows that removing either conflict detection or the trained editor degrades performance.

Load-bearing premise

The load-bearing premise is that the unedited LLM can reliably classify retrieved facts as supporting, contradicting, or unrelated to each reasoning step; if the classifier mistakes an unrelated fact for a contradiction, the editor rewrites a correct chain and the final answer can become wrong, as the paper's Table 9 shows.

Editorial extensions

If this is right

  • If the results hold, the same EditCoT pipeline transfers across task types and languages without prompts being redesigned, covering triple-based multi-hop QA, free-form reasoning updates, and legal statute changes.
  • Knowledge updates would no longer require retraining or weight modification, and the editor can be much smaller than the backbone model, so continuous updates become feasible on modest hardware.
  • Because the method needs no internal model access, it extends to proprietary black-box models such as GPT-4o, with a smaller open model acting as editor.
  • Robustness results suggest the method preserves unrelated knowledge better than baselines and degrades less as batch size grows, which matters for realistic bulk edits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the bottleneck may shift from editing to detection: its own failure analysis shows a false conflict can corrupt an otherwise correct chain, so better retrieval and conflict classification could improve EditCoT more than a stronger editor.
  • One extension beyond the paper is to use the same separation of reasoning path and fact memory to correct other reasoning flaws, not just factual updates, such as outdated procedures or stale instructions.
  • A testable extension would vary the retriever quality while keeping the editor fixed; if gains track retrieval quality more than editor size, that tells practitioners where to invest.
  • The method's dependence on a separately trained editor suggests a modular deployment where a small editor is shared across many large backbones, a configuration the paper only partially explores.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces EditCoT, a non-parametric knowledge-editing framework that updates an LLM by iteratively revising its chain-of-thought. The method first generates an initial CoT and answer, then repeatedly retrieves updated knowledge, detects conflicts between the CoT and the retrieved knowledge using a three-way token-probability classifier, and revises the CoT with a separately trained CoT editor. The editor is trained on synthetically constructed data derived from HotpotQA without human annotation. The authors evaluate EditCoT on MQuAKE-CF-3k, DUNE, and LeKUBE using Llama-3-8B-Instruct, Qwen2.5-14B-Instruct, and GPT-4o, reporting strong performance on DUNE and LeKUBE, competitive results on MQuAKE-CF-3k with Qwen, and superior locality, stability across batch sizes, and inference efficiency relative to several baselines. The paper also includes ablations showing the importance of conflict detection and the trained editor, plus a data-leakage check.

Significance. If the results hold, EditCoT is a meaningful step toward task-agnostic in-context knowledge editing: it does not require retraining at edit time, works across languages and task types, and is applicable to black-box models. The release of code and data, the leakage check in Appendix H, and the component ablations in Table 5 are concrete strengths that increase confidence in the empirical claims. The main weakness is that the headline 'state-of-the-art' claim is not supported by the paper's own comparative tables, and the core conflict-detection module is not directly evaluated. The approach remains a competitive and generalizable baseline that advances the editing-by-reasoning direction, provided the claims are appropriately qualified.

major comments (3)
  1. [Abstract and Section 5.1, Tables 1 and 3] The unqualified claim that EditCoT 'achieves state-of-the-art performance' is internally inconsistent with the reported results. On MQuAKE-CF-3k with Llama-3-8B-Instruct, RAE reaches 54.1% while EditCoT reaches 35.4% (Table 1); on the GPT-4o evaluation, RAE reaches 59.7% while EditCoT reaches 45.0% (Table 3). The 'top performance on 6 out of 8 results' in Section 5.1 is obtained only because RAE is marked not applicable on DUNE and LeKUBE. On the one primary benchmark where a direct comparison is possible, EditCoT is not state of the art. The abstract, introduction, and conclusion should qualify the claim (e.g., 'best among methods applicable to all three benchmarks' or 'competitive with RAE on MQuAKE while extending to new domains').
  2. [Section 3.3.2, Part III] The synthetic data construction assumes that 'the divergence occurs only once in most of the cases' between the RAG-generated CoT and the prefix-guided CoT. This assumption is never validated quantitatively, and it is load-bearing for the correctness of the training tuples: if the two CoTs diverge at multiple steps, then the constructed tuple (Q, CoT_k, rewrite(Step_k), CoT) may not represent a coherent edit, and the editor could learn spurious mappings. The paper should report the proportion of accepted examples where the divergence is indeed unique, and, if multiple divergences occur, analyze whether excluding or transforming them changes the editor's performance on the downstream benchmarks.
  3. [Section 3.2 Step 2 and Table 9] The conflict-detection module is a core component: removing it in Table 5 causes a large drop on MQuAKE-CF (from 40.3 to 32.3 for Llama and from 40.3 to 33.0 for Qwen) and on LeKUBE MCQ (from 58.3 to 48.9). Yet the paper never directly measures the module's precision or recall on the evaluation benchmarks. The failure case in Table 9 shows a false conflict leading to an incorrect edit, and the paper only gives an anecdotal estimate of '1-2 cases out of 200 manually inspected examples.' A systematic evaluation of the conflict detector (e.g., accuracy on held-out conflict/support/unrelated labels sampled from the test sets) would substantiate the claim that the iterative editing loop is reliable and would give readers a concrete sense of when the method is likely to fail.
minor comments (6)
  1. [Section 3.4] The conflict-detection threshold of 0.6 is a hand-set hyperparameter, but no sensitivity analysis is provided. A brief plot or table showing performance as the threshold varies would help establish robustness.
  2. [Appendix J.2] In the conflict-detection prompt, the few-shot assistant responses include both the relevance label and a 'New Thoughts:' continuation. Since the model is supposed to output only the label at test time, the presence of new-thought examples may bias the classifier. Consider removing the 'New Thoughts' portion from the few-shot classification examples or verifying that the probability-based classification is unaffected.
  3. [Figure 4] Figure 4 is split due to a wide horizontal span, which makes it hard to compare all methods. Using a log-scale x-axis or separate subplots with a shared legend would improve readability.
  4. [Table 9] The failure-case description states that the first error is highlighted in red, but the text in the table is not visibly colored. Please use a printer-friendly marker such as bold type or an asterisk.
  5. [Section 7, Limitations] The limitations section mentions higher GPU resource consumption but does not report actual memory or compute numbers. A short quantitative comparison (e.g., peak GPU memory for EditCoT vs. Mello/PokeMQA) would make the trade-off concrete.
  6. [References] LeKUBE is cited as both Wang et al. 2024a and Wang et al. 2024b in the text; please disambiguate these references and use a consistent citation style throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the CoT editor is trained on HotpotQA-derived synthetic data and evaluated on separate knowledge-editing benchmarks; self-citations to LeKUBE/STARD are not load-bearing.

full rationale

The paper's derivation chain is self-contained. The central component, the CoT editor, is trained on automatically constructed tuples from HotpotQA questions (Section 3.3.2) of the form (Q, CoT_k, rewrite(Step_k), CoT), where the 'new knowledge' is extracted from the model's own generated reasoning rather than from the test benchmarks. At inference, the editor is applied to MQuAKE-CF-3k, DUNE, and LeKUBE, whose target answers the editor never sees during training. The conflict-detection threshold of 0.6 (Section 3.4) is a hand-set hyperparameter, not a constant fitted to the evaluated results, and the paper explicitly tests for training-data leakage in Appendix H using an SFT baseline, finding no significant advantage. The failure case in Table 9 further confirms that conflict detection is not tautologically aligned with the benchmark: a false conflict leads to an incorrect edit. The only self-citations of note are to LeKUBE and STARD, which are public benchmark/dataset resources from the same group; they are evaluation resources rather than premises that force the method's design, so they do not constitute load-bearing circularity. The Abstract's unqualified 'state-of-the-art' claim is internally inconsistent with Table 1 (RAE 54.1 vs. EditCoT 35.4 on MQuAKE-CF-3k with Llama-3) and Table 3 (RAE 59.7 vs. EditCoT 45.0 on GPT-4o), but that is a claim-accuracy or correctness issue, not a circular-derivation issue. No step in the claimed derivation reduces by construction to its own input.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method relies on standard LLM inference, a fine-tuned editor, and retrieval. The main hand-chosen parameters are the conflict threshold and training configuration. No new physical or theoretical entities are introduced.

free parameters (3)
  • conflict detection probability threshold = 0.6
    Chosen by hand in Section 3.4 to decide when the three-way classification counts as a conflict; no sensitivity analysis is reported, and the failure case in Table 9 shows a false conflict at this threshold.
  • CoT editor training split and epochs = 80% of generated data, 1 epoch
    Chosen in Section 4.4 without systematic search; Table 6 shows performance is fairly flat across epochs and data sizes, so this setting is not strongly load-bearing.
  • training example counts = 459 (Llama-3-8B), 316 (Qwen2.5-14B)
    Result of filtering HotpotQA-generated data; these counts are small and could affect editor quality, but Table 6 suggests stability beyond 200 samples.
assumptions (4)
  • domain assumption The unedited LLM can reliably detect conflicts between its own CoT steps and retrieved updated knowledge using token-probability classification.
    Used in Step 2 of the workflow; if the classifier misfires, the edit chain is corrupted, as shown in Table 9.
  • domain assumption A CoT editor trained on HotpotQA-derived synthetic conflict data transfers to MQuAKE, DUNE, and LeKUBE without task-specific retraining.
    This is the central generalization claim; only three benchmarks are tested, and the Chinese legal domain shows limited gains.
  • ad hoc to paper During training data construction, the divergence between the RAG-generated CoT and the prefix-guided CoT occurs only once in most cases.
    Stated in Part III of Section 3.3.2; multi-conflict cases are filtered out, so the editor may not learn multi-conflict editing.
  • domain assumption Retrieval from the edited-knowledge memory (Contriever or BM25) returns the relevant pre-edit or post-edit fact at the correct step.
    Step 2 requires step-level retrieval; failure cases in Table 9 show retrieval noise can trigger incorrect edits.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge Editing through Chain-of-Thought." pith.science (2026). https://pith.science/paper/LIJTOWFF

@misc{pith2026241217727,
  author       = {Pith},
  title        = {Pith review of: Knowledge Editing through Chain-of-Thought},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LIJTOWFF}},
  note         = {Machine review of arXiv:2412.17727}
}
read the original abstract

Knowledge Editing is a technique that updates large language models (LLMs) with new information to maintain their world knowledge. This approach avoids the need to rebuild the model from scratch, thereby addressing the high costs associated with frequent retraining. Among these, the in-context editing paradigm stands out for its effectiveness in integrating new knowledge while preserving the model's original capabilities. Despite its potential, existing in-context knowledge editing methods are often task-specific, focusing primarily on multi-hop QA tasks using structured knowledge triples. Moreover, their reliance on few-shot prompting for task decomposition makes them unstable and less effective in generalizing across diverse tasks. In response to these limitations, we propose EditCoT, a novel knowledge editing framework that flexibly and efficiently updates LLMs across various tasks without retraining. EditCoT works by generating a chain-of-thought (CoT) for a given input and then iteratively refining this CoT process using a CoT editor based on updated knowledge. We evaluate EditCoT across a diverse range of benchmarks, covering multiple languages and tasks. The results demonstrate that our approach achieves state-of-the-art performance while offering superior generalization, effectiveness, and stability compared to existing methods, marking a significant advancement in the field of knowledge updating. The code and data of EditCoT are available at: https://github.com/bebr2/EditCoT .

Figures

Figures reproduced from arXiv: 2412.17727 by the authors.

Figure 1
Figure 1. EditCoT updates LLMs by iteratively editing [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Diagram of the dataset construction: Our data source consists solely of questions from HotpotQA, without [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A training data example. The black part of the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Inference time vs. performance: Top-left [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Performance with different batch sizes on [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Performance with different max iterations. A [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: A face-to-face comparison of our proposed EditCoT method with other baseline approaches. The first [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Fine-tuning an LLM with defect detection and utility extraction tasks makes it more robust to noisy, irrelevant, and counterfactual documents in retrieval-augmented generation.

  2. Parametric Retrieval Augmented Generation

    cs.CL 2025-01 conditional novelty 6.0 of 10

    A new RAG paradigm that encodes each document as LoRA parameters and merges retrieved documents' parameter updates into the LLM, outperforming in-context RAG on four QA benchmarks.

  3. Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Teaching an LLM to emit a fixed four-stage reasoning chain during fine-tuning makes single-pass multi-hop knowledge editing robust to distractor facts.

  4. Dynamic and Parametric Retrieval-Augmented Generation

    cs.CL 2025-06 unverdicted novelty 2.0 of 10

    A tutorial outline that categorizes recent RAG work into Dynamic RAG and Parametric RAG, and explains why both are needed.

Reference graph

Works this paper leans on

37 extracted references · 5 canonical work pages · cited by 4 Pith papers

  1. [1]

    Afra Aky \"u rek, Eric Pan, Garry Kuwanto, and Derry Wijaya. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.114 DU n E : Dataset for unified editing . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1847--1861, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.114 2023

  2. [2]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    2020

  3. [3]

    Junjie Chen, Weihang Su, Zhumin Chu, Haitao Li, Qinyao Ai, Yiqun Liu, Min Zhang, and Shaoping Ma. 2024. An automatic and cost-efficient peer-review framework for language generation evaluation. arXiv preprint arXiv:2410.12265

    arXiv 2024

  4. [4]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493--8502, Dublin, Ireland. Association for Computational Linguistics

    doi:10.18653/v1/2022.acl-long.581 2022

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models

    arXiv 2024

  6. [6]

    Hengrui Gu, Kaixiong Zhou, Xiaotian Han, Ninghao Liu, Ruobing Wang, and Xin Wang. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.438 P oke MQA : Programmable knowledge editing for multi-hop question answering . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8069--8083, Bangkok,...

    work page doi:10.18653/v1/2024.acl-long.438 2024

  7. [7]

    Jia - Chen Gu, Hao - Xiang Xu, Jun - Yu Ma, Pan Lu, Zhen - Hua Ling, Kai - Wei Chang, and Nanyun Peng. 2024 b . https://aclanthology.org/2024.emnlp-main.934 Model editing harms general abilities of large language models: Regularization to the rescue . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Mi...

    work page 2024

  8. [8]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. https://openreview.net/forum?id=jKN1pXi7b0 Unsupervised dense information retrieval with contrastive learning . Trans. Mach. Learn. Res., 2022

    2022

Show all 37 references
  1. [9]

    Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.495 Active retrieval augmented generation . In Proceedings of the 2023 Conference on Empirical Metho...

    2023 doi

  2. [10]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html Large language models are zero-shot reasoners . In Advances in Neural Informati...

    2022

  3. [11]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e...

    2020

  4. [12]

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. https://doi.org/10.18653/v1/2023.acl-long.687 Contrastive decoding: Open-ended text generation as optimization . In Proceedings of the 61st Annual...

    2023 doi

  5. [13]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/6f1d43d5a82a37e89b0665b33bf3a182-Abstract-Conference.html Locating and editing factual associations in GPT . In Advances in Neural Information Processing System...

    2022

  6. [14]

    Manning, and Chelsea Finn

    Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. 2022. https://proceedings.mlr.press/v162/mitchell22a.html Memory-based model editing at scale . In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryl...

    2022

  7. [15]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, et al. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report

    2023 arXiv

  8. [16]

    Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval , 3(4):333--389

    2009

  9. [17]

    Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024 a . https://doi.org/10.18653/v1/2024.naacl-short.69 Trusting your evidence: Hallucinate less with context-aware decoding . In Proceedings of the 2024 Conference of the North America...

    2024 doi

  10. [18]

    Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, and Ninghao Liu. 2024 b . https://arxiv.org/abs/2403.19631 Retrieval-enhanced knowledge editing in language models for multi-hop question answering

    2024 arXiv

  11. [19]

    Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Zibing Que, Ning Zheng, Yun Liu, Weixing Shen, and Yiqun Liu. 2024 a . https://arxiv.org/abs/2406.15313 Stard: A chinese statute retrieval dataset with real queries issued by non-professionals

    2024 arXiv

  12. [20]

    Weihang Su, Yichen Tang, Qingyao Ai, Changyue Wang, Zhijing Wu, and Yiqun Liu. 2024 b . Mitigating entity-level hallucination in large language models. arXiv preprint arXiv:2407.09417

    2024 arXiv

  13. [21]

    Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. 2024 c . https://doi.org/10.18653/v1/2024.findings-acl.854 Unsupervised real-time hallucination detection based on the internal states of large language models . In Findings of the Associat...

    2024 doi

  14. [22]

    Weihang Su, Changyue Wang, Anzhe Xie, Qingyao Ai, Yiran Hu, and Yiqun Liu. 2024 d . Legalaid: A large language model for the chinese legal field. https://github.com/oneal2000/LegalAID

    2024

  15. [23]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  16. [24]

    Changyue Wang, Weihang Su, Yiran Hu, Qingyao Ai, Yueyue Wu, Cheng Luo, Yiqun Liu, Min Zhang, and Shaoping Ma. 2024 a . Lekube: A knowledge update benchmark for legal domain. In Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Inf...

    2024

  17. [25]

    Changyue Wang, Weihang Su, Hu Yiran, Qingyao Ai, Yueyue Wu, Cheng Luo, Yiqun Liu, Min Zhang, and Shaoping Ma. 2024 b . https://arxiv.org/abs/2407.14192 Lekube: A legal knowledge update benchmark

    2024 arXiv

  18. [26]

    Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, et al. 2023 a . https://arxiv.org/abs/2308.07269 Easyedit: An easy-to-use knowledge editing framework for large language models . ArXiv preprint, abs/2308.07269

    2023 arXiv

  19. [27]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 b . https://openreview.net/pdf?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confere...

    2023

  20. [28]

    Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, and Yitao Liang. 2024 c . https://arxiv.org/abs/2403.05313 Rat: Retrieval augmented thoughts elicit context-aware reasoning in long-horizon generation . Preprint, arXiv:2403.05313

    2024 arXiv

  21. [29]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning...

    2022

  22. [30]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  23. [31]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, et al. 2024. https://arxiv.org/abs/2407.10671 Qwen2 technical report . ArXiv preprint, abs/2...

    2024 arXiv

  24. [32]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  25. [33]

    Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. 2023. https://doi.org/10.18653/v1/2023.acl-long.320 Verify-and-edit: A knowledge-enhanced chain-of-thought framework . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguist...

    2023 doi

  26. [34]

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.296 Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro...

    2023 doi

  27. [35]

    Zexuan Zhong, Zhengxuan Wu, Christopher Manning, Christopher Potts, and Danqi Chen. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.971 MQ u AKE : Assessing knowledge editing in language models via multi-hop questions . In Proceedings of the 2023 Conference on Empirical Meth...

    2023 doi

  28. [36]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  29. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.