Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Memorization vs. Reasoning: Updating LLMs with New Knowledge

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that continued pretraining on a corpus of realistic news updates lets LLMs memorize new facts but not reason over them, and that memory-conditioned training improves direct recall by up to 25.4 percentage points.

desk verdict Solid benchmark and training method, but the indirect-reasoning headline is not yet supported by the metric as defined. read the letter →

arxiv 2504.12523 v1 pith:ZRPTDVK5 submitted 2025-04-16 cs.CL cs.AIcs.LG

Aochong Oliver Li , Tanya Goyal This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords continuedpretrainingknowledgeupdateconflictmemorizationreasoningmemoryconditionedtrainingbenchmarkchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles the problem of updating a large language model's parametric knowledge after its training cutoff. It introduces Knowledge Update Playground (KUP), an automatic pipeline that simulates realistic knowledge updates by generating fictitious news articles about 1000 entities, together with direct and indirect probes that separate memorization of a new fact from reasoning over its consequences. On KUP, the authors find that every continued pretraining method they benchmark, including data rephrasing, scores under 2% on indirect probes, meaning the models still answer as if the old fact were true. They then propose memory conditioned training (MCT), which prepends self-generated memory tokens to update documents during training, and report direct-probe accuracy gains of up to 25.4 percentage points over continued pretraining baselines. If the results hold, the field's standard way of updating LLMs teaches them to recall new facts without enabling them to apply those facts.

What carries the argument

KUP's curation pipeline: GPT-4O generates candidate mutable facts about 1000 entities, filters them for changeability and objectivity, produces contradictory updated facts with event sequences, and writes five fictitious news articles per update; the base models are probed to confirm they know the old fact and reject the new one. The training corpus combines 3.3M tokens of these evidence documents with 52.4M tokens of real auxiliary news. MCT is the learning mechanism: chunks of self-generated 'memory' completions sampled from the base model are prepended to each evidence document, the loss is masked on those memory tokens so the model focuses on the update, and at inference the model first recalls memory (treated as chain-of-thought) before answering. The direct/indirect probe split is the measuring instrument that lets the authors attribute failures to memorization versus reasoning.

What would settle it

Run the same continued pretraining and MCT pipeline on a corpus of real, verified post-cutoff updates, for example 2024-2025 news events with archived articles, using entities whose old facts the base model knows, and measure indirect probing of old versus new knowledge. If real-update indirect accuracy is substantially above 2% or if the MCT gains disappear, the synthetic KUP updates rather than the learning methods explain the reported results.

Watch

Extended reading notes

Core claim

The central discovery is a separation between memorization and reasoning in model updates. After continued pretraining on KUP's evidence corpus, all tested CPT methods, including standard, rephrased, and memory-conditioned variants, fail at indirect probing, entailing the outdated fact in 78.5% to 83.2% of responses and the updated fact in under 2%. Direct probing tells a different story: MCT, which trains on entity evidence prefixed by chunks of the model's own Wikipedia-style completions about that entity, selects the updated fact over distractors at 60.7% (Llama) and 71.0% (Mistral) in one MCQ setting, up from 20.0% and 17.4% for standard CPT, with gains of up to 25.4 percentage points over the baselines. The paper also finds that memory recall at inference, functionally chain-of-thought, helps knowledge surfacing, and that training perplexity does not predict whether a model will answer correctly.

Load-bearing premise

The load-bearing premise is that GPT-4O-generated fictitious news articles about 1000 entities, filtered so the base model recognizes the old fact and rejects the new one, faithfully reproduce the structure of real post-cutoff knowledge updates; if those synthetic updates are easier or harder than real ones, the under-2% indirect-probe result and the MCT gains may not transfer.

Editorial extensions

If this is right

  • Entity-substitution benchmarks likely overstate how well continued pretraining adapts LLMs to real-world knowledge dynamics, because KUP's updates are mostly contextual rewrites rather than simple attribute swaps.
  • Simply maximizing likelihood on an update corpus is not enough: perplexity on new evidence does not distinguish correct from incorrect MCQ answers, so training objectives need to reward surfacing the updated fact in conflict with old knowledge.
  • MCT is a practical, lightweight addition to CPT pipelines: it requires only self-generated memory chunks and a masked loss, yet yields the largest direct-probe gains when combined with inference-time memory recall.
  • Chain-of-thought prompting helps a knowledge-intensive task here, suggesting that CoT or memory recall can resolve parametric memory conflicts even when the task is not a traditional reasoning benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the under-2% indirect-probe result holds, the bottleneck is not storing the updated fact but choosing it over the old fact during generation; retrieval-free update methods may need an explicit conflict-resolution step before they can match RAG on indirect questions.
  • Because KUP uses fictitious updates, the 25.4-point MCT gain may not transfer to real post-cutoff corpora; testing MCT on archived real news would separate method quality from synthetic-corpus ease.
  • The memory-token trick suggests a general recipe: any CPT method could elicit entity-specific context from the base model before training, and the same elicitation at inference could be chained with retrieved passages to push indirect accuracy.
  • The 60-question indirect set is small and manually curated; an automated generator of indirect probes over more entities would be needed to confirm the catastrophic failure rate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Knowledge Update Playground (KUP), a pipeline that generates synthetic but realistic knowledge updates (f_old -> f_new) for 1000 entities, along with an evidence corpus of fictitious news articles and auxiliary scraped news. It also proposes memory conditioned training (MCT), which prepends self-generated memory tokens during continued pretraining and uses a memory-recall step at inference. Experiments on Llama-3.1-8B and Mistral-7B-v0.3 compare MCT with standard CPT and rephrased CPT, including a no-train baseline and RAG upper bounds. The paper claims that all CPT methods fail indirect probing (<2% 'accuracy'), that MCT significantly improves direct-probing memorization (up to 25.4 percentage points), and that KUP is a challenging benchmark for future work.

Significance. The paper has several strengths: the direct-probing experiments are carefully controlled, with f_old/f_new verified against the test models, 1% replay data added to all training runs, chain-of-thought applied uniformly, and both a no-train baseline and RAG upper bounds reported. The KUP dataset and code are released, and the manual analysis of CoT traces provides useful evidence about what MCT recalls. If the central claims were fully supported, the paper would make a valuable contribution to knowledge-update evaluation. However, the headline '<2% indirect probing' conflates entailment classification with answer accuracy, and the paper's own Table 7 contains a number that contradicts the '<2%' wording. The 'memorization but not reasoning' conclusion therefore needs re-analysis before the paper's main message can be accepted.

major comments (4)
  1. [Abstract; §4.1.2; Table 7] The headline claim that 'the best CPT models achieve <2% in indirect probing' is not supported as an accuracy claim. Table 7 reports only the fraction of responses entailing UPD, OLD, or N/A, and the N/A category (14.7–22.0%) includes responses that do not mention the target entity. For a list-style question such as 'What events does Baillie Gifford still sponsor?', an answer that correctly omits the no-longer-sponsored entity is scored N/A rather than UPD. No human-validated correctness score is reported for the 60-question indirect set, and the RAG oracle row is scored under the same N/A rubric, so it does not independently validate the metric. In addition, Table 7 shows 3.0% UPD for Mistral Re+CoT, contradicting the abstract's 'all CPT approaches' / '<2%' wording. Please re-analyze the indirect set with human annotation and distinguish answer correctness from entailment of the update.
  2. [§5.2; Table 7] The indirect-probing protocol is underspecified with respect to the supervised fine-tuning step. The text says that for indirect probing the authors 'supervised fine-tune M>T' on the 4.2K direct-probing Q&A pairs, but it does not state whether the CPT and Re+CoT baselines received identical SFT or whether only the MCT-trained model was fine-tuned. If the training histories differ, the comparison in Table 7 is not controlled. Please specify the exact protocol for each row and, if necessary, re-run with identical SFT for all methods.
  3. [§4.1.1; Table 6] The free-form direct-probing evaluation provides the updated fact statement f_new to the model as context at test time. Table 6 therefore measures grounded question answering given the update statement, not closed-book memorization of the update from the evidence corpus. Since the paper uses Table 6 to argue that MCT outperforms baselines in direct probing, the authors should either evaluate without exposing f_new or explicitly state that Table 6 measures contextualized QA rather than memorization.
  4. [§2.1; §9] The external validity of the 'realistic knowledge updates' claim rests on GPT-4O-generated fictitious updates and news articles. The only validation against real knowledge dynamics is the 50-point manual annotation in Table 2, which measures the presence of narrative properties rather than whether the synthetic updates are representative of actual post-cutoff knowledge changes. Given that the paper's conclusion about CPT methods on realistic evidence corpora depends on this premise, I would like to see either a comparison on real evolving knowledge (e.g., Realtime QA or GROWOVER) or a human evaluation of update plausibility and difficulty. The limitation section acknowledges the issue but does not resolve it.
minor comments (4)
  1. [§4.1.2] The sentence 'Note that there may exist cases where the model generation does mention entity e, and therefore, cannot be classified as entailing either' should say 'does not mention entity e'; as written it states the opposite of the intended meaning.
  2. [Figure 2 and §2.1] There is a typo: 'axillary documents' should be 'auxiliary documents'.
  3. [Abstract; §1] The model name is written inconsistently as 'LLama-3.1-8B' and 'LLama'; the correct spelling is 'LLaMA-3.1-8B'.
  4. [Conclusion] In the conclusion, 'systemically study' should be 'systematically study'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; KUP and MCT are independently defined, and MCT is evaluated against baselines under the same inference procedure.

full rationale

The derivation chain is self-contained. KUP's update pairs are generated and verified independently of the downstream model answers; the evidence corpus is produced from event sequences, and the MCQ/free-form questions are constructed from metadata withheld from the training corpus, so the benchmark is not co-defined with MCT. MCT's memory tokens are sampled from the base model before training, and at inference memory recall is applied as CoT to MCT and baselines alike (Section 3: “we also apply it to other baseline methods during inference”), so the 25.4-point direct-probing gain is not a relabeled fit. There are no load-bearing self-citations: the cited baselines and prior benchmarks are external works. The only concern is the abstract's “<2% accuracy” phrasing: it reports the UPD entailment fraction from Table 7, and N/A responses (14.7–22.0%) include answers that omit the entity, so the indirect metric is a surfacing measure rather than a validated correctness measure. That is a construct-validity and correctness caveat, not a circular reduction: nothing in the paper's equations forces the low UPD number once the metric is specified, and the high OLD entailment rates (78.5–83.2%) are an empirical finding. Therefore no circularity is exhibited.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on synthetic validity and LLM-judge reliability rather than on a closed-form derivation, so the ledger records the hand-set dataset-construction thresholds and domain assumptions. There are no invented physical entities.

free parameters (6)
  • Entity filtering ROUGE-2 threshold = 0.1
    Used in §2.1 Step 0; hand-chosen to keep entities whose Wikipedia-style completion overlaps the real article, and this directly shapes the 1000-entity set.
  • Updates retained per entity = 1
    After verification, only one f_old-to-f_new pair is kept per entity to avoid contradictory updates (§2.1).
  • Number of evidence articles per update = 5
    GPT-4O generates five news articles per update pair (§2.1 Step 2); this choice sets the evidence corpus size.
  • Indirect probe test set size = 60
    Manually curated questions in §4.1.2; also filtered to entities where both models answered the MCQs correctly.
  • Learning rate = 1e-05
    Training hyperparameter in §C used for all methods; a hand choice, though shared across methods so it does not drive relative comparisons.
  • Replay data proportion = 1%
    REDPAJAMA replay data included for all methods to limit catastrophic forgetting (§4).
assumptions (5)
  • domain assumption GPT-4O-generated fictitious updates and news articles are realistic proxies for real-world knowledge dynamics.
    The whole benchmark and the paper's conclusions depend on this; §2.1 generates updates and articles with GPT-4O, and §2.2 only provides qualitative annotation as evidence.
  • domain assumption True/False probing of the test models correctly establishes that f_old is in parametric knowledge and f_new contradicts it.
    Used in §2.1 for verification and filters roughly 30% of update tuples; any noise here propagates to all later measurements.
  • domain assumption GPT-4O-mini judges free-form and indirect response entailment accurately.
    All free-form and indirect scores in Tables 6 and 7 come from this judge with no human agreement reported.
  • ad hoc to paper Auxiliary web articles do not mention the fictitious updates.
    Real news collected via SERPHouse is added for realism (§2.1 Step 2); if any auxiliary article leaks an update, test answers could be inflated.
  • domain assumption Memory-recall/CoT at inference is equally applicable to all baselines.
    The paper states CoT is applied to baseline methods in §3, but the exact prompt is not fully specified in the main text, so prompt differences could advantage MCT.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memorization vs. Reasoning: Updating LLMs with New Knowledge." pith.science (2026). https://pith.science/paper/ZRPTDVK5

@misc{pith2026250412523,
  author       = {Pith},
  title        = {Pith review of: Memorization vs. Reasoning: Updating LLMs with New Knowledge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZRPTDVK5}},
  note         = {Machine review of arXiv:2504.12523}
}
abstract

Large language models (LLMs) encode vast amounts of pre-trained knowledge in their parameters, but updating them as real-world information evolves remains a challenge. Existing methodologies and benchmarks primarily target entity substitutions, failing to capture the full breadth of complex real-world dynamics. In this paper, we introduce Knowledge Update Playground (KUP), an automatic pipeline for simulating realistic knowledge updates reflected in an evidence corpora. KUP's evaluation framework includes direct and indirect probes to both test memorization of updated facts and reasoning over them, for any update learning methods. Next, we present a lightweight method called memory conditioned training (MCT), which conditions tokens in the update corpus on self-generated "memory" tokens during training. Our strategy encourages LLMs to surface and reason over newly memorized knowledge at inference. Our results on two strong LLMs show that (1) KUP benchmark is highly challenging, with the best CPT models achieving $<2\%$ in indirect probing setting (reasoning) and (2) MCT training significantly outperforms prior continued pre-training (CPT) baselines, improving direct probing (memorization) results by up to $25.4\%$.

Figures

Figures reproduced from arXiv: 2504.12523 by the authors.

Figure 1
Figure 1. Example of LLM that is continued pre-trained [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our KUP data curation pipeline. We omit details about verification of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of Memory Conditioned Training [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can a Language Model Learn Facts Continually in Its Weights?

    cs.CL 2026-07 unverdicted novelty 7.0 of 10

    Weight writes create question-keyed knowledge that later sequential writes redirect, so context—not weights—is the reliable channel for facts that must survive or be composed.

  2. ScienceMeter: Tracking Scientific Knowledge Updates in Language Models

    cs.CL 2025-05 reject novelty 6.0 of 10

    ScienceMeter evaluates language model knowledge updates across three axes, preservation of old scientific claims, acquisition of new claims, and projection to future findings, and finds all current methods fall short.

Reference graph

Works this paper leans on

45 extracted references · 40 canonical work pages · cited by 2 Pith papers

  1. [1]

    Misleading choices also should include more details, such as number, names, locations, than the correct choice

    All misleading choices (B, C, D) should be longer than the correct choice in length. Misleading choices also should include more details, such as number, names, locations, than the correct choice. But do not introduce any specific date details

    work page

  2. [2]

    Dayoon Ko, Jinyoung Kim, Hahyeon Choi, and Gun- hee Kim

    Realtime qa: What’s the answer right now? Preprint, arXiv:2207.13332. Dayoon Ko, Jinyoung Kim, Hahyeon Choi, and Gun- hee Kim. 2024a. Growover: How can llms adapt to growing real-world knowledge? arXiv preprint arXiv:2406.05606. Dayoon Ko, Jinyoung Kim, Hahyeon Choi, and Gunhee Kim. 2024b. GrowOVER: How can LLMs adapt to growing real-world knowledge? In P...

    arXiv 2025

  3. [3]

    Always refer to the entity by its name rather than using pronouns

    work page

  4. [4]

    Preprint, arXiv:2410.20796

    Rephrasing natural text data with different lan- guages and quality levels for large language model pre-training. Preprint, arXiv:2410.20796. Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2022. Elle: Efficient lifelong pre-training for emerging data. Preprint, arXiv:2203.06311. ANTHONY ROBINS. 1995. Catastrophic for...

    arXiv

  5. [5]

    Today’s Date: December 2023

    ConflictBank: A benchmark for evaluating the influence of knowledge conflicts in LLMs. In Advances in Neural Information Processing Systems. Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, and Thang Luong. 2024. Fresh- LLMs: Refreshing large language models with search engine augmentati...

    arXiv 2024

  6. [7]

    All choices (A, B, C, D) should strictly use the same time tense and be written in the same sentence structure to be stylistically indistinguishable

    work page

  7. [9]

    content” and “role

    Do not include additional comments after the question Question: Which of the following about {entity} is True? Table 12: Prompt for generating Update vs. Distractors and Update vs. Old MCQ E.4 Prompt for Generating Fictitious News We use the prompt in Table 21 to generate five different audience groups for each news article. We use the prompt in Table 22 ...

    work page

  8. [10]

    the person,

    Self-contained questions: Each question must be understandable without requiring the article as context. Each question should include specifics such as names, dates, events, or changes. Avoid anaphoric or vague noun phrases, like “the person,” “the article,” “the event,” “the transition” etc. Readers cannot access the article content nor know what transit...

    work page

Show all 45 references
  1. [11]

    Do not assume the reader has seen previous questions

    Independent questions: Each question must stand alone and will be presented individually. Do not assume the reader has seen previous questions. Avoid referencing other questions or relying on their background for context. Each question should be fully self-explanatory

  2. [12]

    Diversity of questions: Generate 20 distinct and meaningful questions covering different key aspects of the article

  3. [13]

    Supported answers: Each answer must be correct and grounded in the article, providing supporting evidence or key details

  4. [14]

    If quotation marks are necessary within content, use single quotes (”) instead

    Avoiding Quotation Marks: Ensure all double quotes inside JSON values are properly escaped to prevent syntax errors in Python. If quotation marks are necessary within content, use single quotes (”) instead. Additional Instructions:

  5. [15]

    Change-oriented question: Given that the article focuses on recent changes, include 1 to 3 simple questions that elicit answers contrasting before and after the change naturally

  6. [16]

    For example, an answer should explain what was true before the change, when the change occurred, and how the fact evolved into its new state

    Contextualized answer: For change-oriented questions, ensure answers describe both the previous and updated states of the entity. For example, an answer should explain what was true before the change, when the change occurred, and how the fact evolved into its new state

  7. [17]

    MoMa is free for full-time students from Columbia University and CUNY schools,

    You do not need to differentiate these Q&A pairs from others. Include all questions in the same list of JSON objects. Table 13: Prompt for Free-form QA MODEL TRAINING OLD PERPLEXITY UPDATE PERPLEXITY ✓ ✗ ✓ ✗ LLAMA CPT 11.95 11.72 4.66 4.71 MCT 11.41 12.06 4.29 4.30 REPHRASE 11...

  8. [18]

    Avoid previous fact, past results, or accomplishments that cannot be any different in the future

    Current Status: Focus on the entity’s current realities. Avoid previous fact, past results, or accomplishments that cannot be any different in the future

  9. [19]

    Exclude very stable attributes that are unlikely to change or require unrealistic assumptions for change

    Changeable: Suggest facts that are likely to change in the future under reasonable and realistic circumstances. Exclude very stable attributes that are unlikely to change or require unrealistic assumptions for change

  10. [20]

    Avoid subjective opinions, speculative commentary, or obscure and vague answers

    Objective & Detailed: Facts must be objective, detailed, and universally agreed upon. Avoid subjective opinions, speculative commentary, or obscure and vague answers

  11. [21]

    actively,

    Avoid descripitive adverbs such as "actively," "frequently," or "currently" in the fact statement First, I will show you some examples Category: people Entity: Yo-Yo Ma facts = ["Yo-Yo Ma is performing on international concert tours", "Yo-Yo Ma records music under the Sony Cla...

  12. [22]

    there exists nonfactual information in bad statements

    Factual: all details in good statements are truthful vs. there exists nonfactual information in bad statements

  13. [23]

    bad statements, which might use present tense, describe past reality or achieved results that are not subject to possible changes

    Temporal: good statements describe the current status of the entity vs. bad statements, which might use present tense, describe past reality or achieved results that are not subject to possible changes

  14. [24]

    Changeable: good statements are subject to be invalidated by reasonable events in the future; bad statements are established realities that cannot be changed under most any circumstance

  15. [25]

    Label: good

    Objective: good statements are absolutely objective and not opinionated vs. bad statements are subjective or commentary I will show you some good statements first. a. Rupi Kaur is currently publishing new poetry books with Andrews McMeel Publishing. b. The current title sponso...

    1991

  16. [26]

    New York Yankees

    Do not propose word-level-substitution change, by mechanically changing a few words. For example, if the entity is "New York Yankees", changing "Aaron Boone is the team’s field manager" to "As of 2025, Sarah Thompson serves as New York Yankees’ field manager" essentially repla...

    2025

  17. [27]

    Emma Watson has been involved in various sustainable fashion projects

    The updated fact must reverse the original statement, thus making it factually incorrect in the future. The focus is on the entity. Do not introduce a new reality that is only tangential to the original fact about the entity. For example, if the fact is "Emma Watson has been i...

  18. [28]

    Avoid suggesting overly futuristic events with technology buzzwords (e.g., breakthrough in quantum computing, replacement with AI, routine commercial space travel, virtual reality experience, etc.)

  19. [29]

    retirement,

    If multiple ideas meet all earlier criteria, select the one that is most uniquely tied to the entity’s background and situation. Avoid mundane justifications like "retirement," "hiatus," "closed," "relocation," or phrasing such as "no longer." Also avoid reasons citing "transi...

  20. [30]

    This fact is not changeable

    The update statement should be specified with fine-grained details. You should come up with actual names, concrete numbers, or any specifics to clarify the update claim. Note: I want high-quality and very realistic change. If you cannot find updates that satisfy all criteria, ...

  21. [31]

    Audience Group: Identify a specific target audience and explain the language, tone, and writing styles that would best resonate with them

  22. [32]

    In each guideline, specify these concrete details in one or two sentences

    Event Details: The event statement have many missing details such as person names, dates (between 2025 to 2027), locations, numerical information in the event statement. In each guideline, specify these concrete details in one or two sentences. Ensure that the details across a...

    2025

  23. [33]

    Separate each writing guideline with a line containing three dashes (—)

  24. [34]

    Do not number or index the guidelines

  25. [35]

    Do not include extra comments or explanations outside of the guidelines. Entity: {entity} Event: {update} Claim: {fact} Table 21: Prompt for generating event sequence and audience group for news articles Based on the provided statement, craft a realistic and coherent news repo...

    2025

  26. [36]

    The content is too empty, sparse, and lacks detail

  27. [37]

    The writing style sounds very artificial and overly synthetic

  28. [38]

    The article is poorly structured and does not have a focus for its target audience

  29. [39]

    Instruction:

    It does not include specific details, like names, numbers, data, etc., in many parts of the article. Instruction:

  30. [40]

    You should very closely emulate the natural writing style, density of details and information, and language style found in the Article Excerpt

  31. [41]

    However, do not change the core of the original article: {update}

    You should use the same article structure (both beginning and body paragraphs of the excerpt article), storytelling approach, and article format as the Article Excerpt. However, do not change the core of the original article: {update}

  32. [42]

    Date:",

    Avoid using any explicit markers or headings (e.g., "Date:", "Headline:", "Title:", or "Section:")

  33. [43]

    Any new information must not contradict the original AI-generated article

    You can introduce any additional details, such as specific names, numbers, and data, where appropriate, to make the article richer and more informative. Any new information must not contradict the original AI-generated article

  34. [44]

    If the Article Excerpt is not in English, you must still craft the refined article in English

  35. [45]

    {excerpt}

    Target {audience}. You should add additional concrete details, beyond original content, tailored to this group of readers Article Excerpt: "{excerpt}" Table 23: Prompt for generating fictitious news articles from base news articles 21

  36. [2021]

    pages 7052–7063, Online and Punta Cana, Dominican Republic

    Entity-based knowledge conflicts in ques- tion answering. pages 7052–7063, Online and Punta Cana, Dominican Republic. Zimu Lu, Aojun Zhou, Ke Wang, Houxing Ren, Weikang Shi, Junting Pan, Mingjie Zhan, and Hong- sheng Li. 2024. Mathcoder2: Better math reasoning from continued p...

    2024 arXiv

  37. [2024]

    Dated data: Tracing knowledge cutoffs in large language models. CoRR. Bosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Luu Anh Tuan, and Shafiq Joty. 2024. Data augmentation using llms: Data perspectives, learning paradigms a...

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.