{"id":"265b3be6-3327-4e0b-bed9-ca74b7687001","arxiv_id":"2412.07781","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper defines faithfulness as reproducibility and reports moderate to high reproducibility for LLM-generated layperson algorithms in law, finance, and health, but the algorithms are not shown to reflect the models' internal mechanisms.","lead":"This paper introduces 'ReQuesting', a prompt-based method that asks LLMs to write a plain-language algorithm for a task and then checks how often the model reproduces its own answers when told to follow that algorithm. The authors report high overall reproducibility across legal, financial, and health tasks, and interpret this as evidence that the algorithms faithfully represent the model's reasoning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The faithfulness claim is unsupported: no control algorithm is tested, and the paper's own stricter metric, PreRR, shows only moderate agreement, contradicting the 'high degree of faithfulness' claimed.","rationale":"The paper proposes an interesting prompting framework with potential as a pilot study, but the central claim that extracted algorithms are faithful representations of LLM reasoning rests on two insecure steps. First, Section 2.1 defines faithfulness as reproducibility without any control condition, so high reproducibility may simply reflect output consistency under similar prompts rather than the algorithm's content. Second, even by the paper's own stricter metric, PreRR, agreement is often moderate (0.39-0.73 across Tables 2-4), while the abstract claims 'high degree of reproducibility' based on PerRR, a metric that Appendix G shows can give near-perfect values for very different prediction sets at low Macro-F1. The Limitations section explicitly acknowledges that prompt template and context influence reproducibility and were not controlled, which is precisely the confound a generic-algorithm control would address. The RQ3 'intrinsic reasoning' analysis is preliminary, manual, and based on a much smaller model (LLaMA 3.2-1B) than the models used in the main experiments, so it cannot independently support the faithfulness claim. None of this is an ad hominem; the methodological gap is concrete and testable. If the proposed control experiment were run and showed a clear separation between A and a generic algorithm, the paper would have a much stronger case. As it stands, the evidence does not support the faithfulness claim, so the reader's REJECT verdict is appropriate and no adjustment is needed.","tokens_in":26223,"tokens_out":5049,"duration_ms":45345,"concrete_test":"Run the ReQuest protocol on each task/LLM combination, but replace the ReQuest algorithm A with two control conditions: (i) a generic algorithm ('Read the input, think carefully, and output the labels you consider most likely'), and (ii) a mismatched algorithm from a different task/domain (e.g., use the health algorithm for statute prediction). Compute PerRR and PreRR between the task-prompt outputs and the control-algorithm-prompted outputs. If the control conditions achieve PerRR/PreRR values statistically indistinguishable from those obtained with A, then reproducibility is not attributable to A's content and the faithfulness claim collapses. Additionally, report PreRR alongside PerRR for all conditions, since Appendix G demonstrates that PerRR alone can be near 100% for prediction sets with only 0.66 Jaccard overlap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim equates reproducibility with faithfulness: Section 2.1 asserts that if the Robustness Check Prompt produces comparable results to the Task Prompt, then algorithm A may be considered a faithful representation of the LLM's internal mechanism. This equivalence is never tested against a control condition. Without a generic or mismatched algorithm as a baseline, the high PerRR scores in Tables 2-4 may simply reflect that the same LLM, given the same task and told to 'strictly follow' any plausible steps, produces similar Macro-F1 outputs. Indeed, the ReQuest algorithms shown in Tables 8, 9, 18, 19, 23, 26, and 29 are generic procedural descriptions that the model would likely follow implicitly even without being prompted. The Limitations section concedes that prompt template and context influence reproducibility and were not controlled. Furthermore, the paper's own stricter metric, PreRR (Jaccard overlap of exact predictions), is often modest: e.g., Table 2 statute prediction PreRR values of 0.52, 0.61, 0.45, 0.39, and 0.41; Table 4 health PreRR values of 0.62, 0.70, 0.67, 0.99, 0.58, and 0.61. Appendix G shows PerRR can be near 100% while PreRR is 0.66 for very different prediction sets at low F1. Thus the 'high degree of reproducibility' headline relies on a metric the paper itself demonstrates can be misleading, and the faithfulness inference lacks any baseline that would rule out trivial prompt-similarity explanations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'ReQuesting,' a prompting regime in which an LLM first performs a task (Task Prompt), then is asked to produce a layperson-understandable algorithm for the task (ReQuest Prompt), and then is asked to execute that algorithm (Robustness Check Prompt). Faithfulness of the algorithm to the LLM's internal mechanism is operationalized as reproducibility between the Task Prompt outputs and the Robustness Check outputs, measured by PerRR (ratio of Macro-F1 scores) and PreRR (Jaccard overlap of per-instance predictions). The framework is evaluated on legal statute prediction, human rights violation prediction, financial stock movement prediction, and mental-health classification tasks using gemini-1.0-pro and llama3-70b in intra- and inter-LLM setups. A third research question explores whether the generated algorithms align with the models' intrinsic reasoning by examining decoding paths on a smaller LLaMA model. The paper concludes that ReQuesting achieves faithful generation of explainable, layperson-understandable algorithms through high reproducibility.","tokens_in":26556,"tokens_out":5332,"duration_ms":48717,"significance":"If the central claim were established, the paper would offer a low-cost, prompt-based method for producing human-readable explanations of LLM behavior in high-stakes domains, with clear practical appeal for laypersons and domain professionals. The paper also contributes a concrete evaluation setup (Prompt/ReQuest/Robustness), two reproducibility metrics (PerRR and PreRR), and a large set of documented prompts for three domains. The manuscript's main strength is the breadth of the empirical exploration across four tasks and two proprietary/open models, and the transparency about the exact prompts used. However, the conceptual gap between 'reproducibility' and 'faithfulness to internal mechanism' is not bridged, and the paper's own data contain substantial portions (e.g., low PreRR values) that undercut the strong headline claim. The study is a useful exploratory report, but as it stands it does not provide evidence for the central claim.","major_comments":[{"comment":"The paper defines faithfulness as reproducibility: 'if the Robustness Check Prompt produces comparable results with the original LLM prompt (Task Prompt), then A may be considered a faithful (and explainable) representation of the complex mechanism.' This is a definitional stipulation, not an empirically established relationship. The paper then uses high reproducibility as evidence of faithful generation (e.g., in the abstract and the caption of Figure 3), making the central claim largely a restatement of the chosen metric. There is no control condition, such as executing a generic or mismatched algorithm or a scrambled version of A, to show that reproducibility is specific to the content of the generated algorithm. Without such a control, the high PerRR values could simply reflect the LLM's output consistency under similar prompts.","section":"Section 2.1"},{"comment":"The paper's own stricter metric, PreRR, shows only moderate agreement in many configurations: e.g., Table 2 statute-prediction PreRR values are 0.5188, 0.4487, 0.6083, 0.3880, and 0.4103; Table 4 health PreRR values are 0.612, 0.701, 0.672, 0.986, 0.657, and 0.605. Yet the abstract and Figure 3 describe the results as 'high reproducibility' based on PerRR. Appendix G demonstrates a toy case where PerRR is 100% while PreRR is 0.66, explicitly showing that high PerRR does not imply prediction-level reproducibility. Thus the headline claim relies on a metric the paper itself demonstrates can be misleading, and the more faithful metric (PreRR) does not support the strong conclusion.","section":"Tables 2-4, Appendix G"},{"comment":"The RQ3 alignment study uses LLaMA 3.2-1B, not the gemini-1.0-pro and llama3-70b models used in the main experiments, and the paper acknowledges that 'the performance of which may not be comparable to the candidate LLMs.' The evaluation is based on manual interpretation ('In the absence of an established method, we manually verify and interpret'), with no quantitative agreement measure between the ReQuest algorithm and the intrinsic reasoning. Table 5 shows only a single anecdotal example. This evidence cannot support the claim that the ReQuest algorithms align with the intrinsic reasoning of the main models.","section":"Section 6, Table 5"},{"comment":"The Limitations section concedes that 'factors like the prompt template and context may influence reproducibility' but does not test the magnitude of this influence or compare the ReQuest regime against a baseline without any algorithm. Similarly, Section 5.1 states that 'No significant performance differences were observed across multiple runs' without reporting standard deviations, confidence intervals, or the number of runs. This lack of variance information makes it impossible to assess the stability of the reproducibility scores that are the core evidence.","section":"Limitations and Section 5.1"}],"minor_comments":[{"comment":"There is a typo in the abstract: 'alignent' should be 'alignment.'","section":"Abstract"},{"comment":"The PerRR formula is ambiguous due to missing parentheses; it is unclear whether the expression is 100 - (|diff|)/l1*100 or (100 - |diff|/l1)*100. Please clarify the mathematical notation.","section":"Section 2.1.1"},{"comment":"The arrows (↑ and ↓) in the result tables are not explained; the caption should state what the direction of the arrow indicates (e.g., improvement or degradation relative to the Task Prompt baseline).","section":"Tables 2-4"},{"comment":"There is a typo in the Robustness Check Prompt: 'Anyting else at the end' should be 'Anything else at the end.'","section":"Table 20"},{"comment":"The paper does not state the test-set sizes for several of the datasets (e.g., the number of test instances for the statute prediction, stock prediction, and health tasks), which is important context when interpreting Macro-F1 and Jaccard-based scores.","section":"General"},{"comment":"The reference to 'Figure 5' and 'Figure 6' in the text is somewhat vague; the figures should be captioned more explicitly so the distribution plots can be tied to the claims about PerRR and PreRR behavior.","section":"Appendix G"}],"recommendation":"major_revision","confidential_remarks":"The paper reports a large number of experiments and makes its prompts available in appendices, which is commendable. However, the central claim—that high reproducibility implies faithful representation of the LLM's internal mechanism—is not operationalized in a way that prevents the conclusion from being an artifact of the chosen metric. A revision that adds control algorithms, emphasizes PreRR as the primary metric, and clearly labels the contribution as 'a reproducibility benchmark for prompt-based algorithm generation' rather than 'faithful mechanism extraction' would be much stronger and would align the claims with the actual evidence. The paper is within scope for a CS-HC venue, but the current framing is likely to mislead readers in high-stakes application domains."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: the ReQuesting approach is a reasonable way to measure whether an LLM can execute its own self-described procedure, but the central claim — that the extracted algorithm is a faithful representation of the model's working mechanism — is an unsupported jump. The paper's stricter metric, PreRR, actually shows the reverse on several tasks.\n\nWhat's new: the prompt regime is genuinely fresh. After the task prompt, the model is asked to produce a deterministic algorithm; that algorithm is then replayed in a \"robustness check\" prompt. The paper defines two metrics, PerRR (Macro-F1 reproduction ratio) and PreRR (Jaccard overlap of exact predictions), and applies them in intra- and inter-LLM setups across law, finance, and health. The appendix includes full prompts and algorithms, plus a worked example showing why PerRR alone can mislead. That is a useful experimental design for studying output consistency.\n\nThe soft spot is the semantic leap in Section 2.1: \"if the Robustness Check Prompt produces comparable results with the original LLM prompt, then A may be considered a faithful representation of the complex mechanism.\" Faithfulness is defined as reproducibility, then high reproducibility is reported as evidence of faithfulness. No control condition — no random, generic, or mismatched algorithm — rules out the possibility that the model simply produces similar outputs under similar prompt templates regardless of algorithm content. The paper's own PreRR numbers are moderate to low on several tasks (e.g., 0.34 for Amazon, 0.39–0.52 for statute prediction), and Section 5.2 concedes that \"PreRR values are notably lower than PerRR values across the board... This would suggest lack of deterministic reasoning at the individual data point level.\" That sentence directly contradicts the abstract's \"high degree of faithfulness.\" The Limitations section also admits prompt template and context were not controlled. The RQ3 alignment study is a pilot at best: a 1B model, k=10, and manual verification on one example.\n\nWho is this for? People interested in prompt-based explainability or reproducibility benchmarking could find the ReQuesting regime a useful starting point, and the metric discussion is a reasonable caution against trusting aggregate scores. But as a demonstration of faithful self-explanation, the paper overreaches.\n\nMy recommendation: send it to peer review, but expect major revision. The core prompting idea deserves a serious look. The paper needs a control baseline, a toned-down abstract, and framing of PerRR/PreRR as consistency measures rather than faithfulness measures. If the authors add a generic-algorithm control and reposition the results as reproducibility without claiming mechanism capture, this could become a useful paper. Desk reject is not warranted; it deserves referee time.","headline":"A sensible prompt-based consistency measure, but the paper's leap from reproducibility to 'faithful self-explanation' is not supported by its own data.","tokens_in":27089,"tokens_out":3064,"would_cite":false,"duration_ms":30512,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces ReQuesting, a prompting technique that asks an LLM to write out a plain-language algorithm of how it answered a task, and claims the algorithm is faithful when it reproduces the model's original outputs.","keywords":["LLM explainability","reproducibility","layperson-understandable algorithms","ReQuesting","faithfulness","high-stakes domains","prompt engineering","zero-shot classification"],"falsifier":"Run the ReQuest pipeline but replace the extracted algorithm with a placebo or deliberately corrupted one, such as a generic 'classify by your best judgment' instruction or the same steps in shuffled order, and compare the PerRR/PreRR values. If the placebo achieves reproducibility scores comparable to the real algorithm, the metric tracks prompt-recognition rather than the algorithm's content, and the faithfulness claim fails.","tokens_in":26054,"feed_emoji":"🧠","tokens_out":9915,"duration_ms":81425,"temperature":0.7,"pith_summary":"The paper introduces ReQuesting, a three-prompt technique that asks an LLM to (1) perform a task, (2) write out a plain-language, step-by-step algorithm that it says it used, and (3) redo the task by strictly following that algorithm. The central claim is that when the algorithm-executed outputs closely reproduce the original task outputs, the algorithm is a faithful, layperson-understandable explanation of the model's working mechanism. The authors test this on legal, financial, and mental-health classification tasks using two large language models and report high reproducibility scores, with most PerRR values above 80% and several near 98%, which they read as evidence of faithfulness. The significance is practical: if correct, non-experts could obtain a verifiable, human-readable account of what an LLM does in high-stakes decisions without access to the model's weights or internals.","feed_headline":"LLMs can write plain-language algorithms for their own decisions","feed_subtitle":"ReQuesting asks the model to write its own steps, then checks if those steps reproduce its answers.","key_machinery":"The central mechanism is the ReQuest prompt regime, composed of the Task Prompt, the ReQuest Prompt that extracts a natural-language algorithm A, and the Robustness Check Prompt that instructs the model to act as a bot that strictly follows steps and execute A. Faithfulness is measured by two reproducibility scores: PerRR, the percentage of the macro-F1 score that is retained when comparing task-prompt outputs to algorithm-run outputs, and PreRR, the average Jaccard overlap of the exact predicted label sets for each test instance. The operational identification of faithfulness with reproducibility is the move that carries the argument: the metrics make that identification numerically testable.","core_discovery":"The authors find that a three-prompt regime, called ReQuesting, can elicit a natural-language algorithm from an LLM and that executing that algorithm reproduces the original task output closely enough to be considered faithful. For each task, the model first answers a Task Prompt; a ReQuest Prompt then asks it to write the steps it followed and to make those steps deterministic; a Robustness Check Prompt has the same or another model execute those steps. Faithfulness is defined operationally as reproducibility: if the macro-F1 performance and the exact per-instance predictions of the algorithm-run match the original task-run closely, the algorithm is considered a faithful representation of the original mechanism. Across statute prediction, human-rights violation prediction, stock movement prediction, and two mental-health classification tasks, the paper reports PerRR (percentage macro-F1 reproduction) values mostly 82–99% for intra-LLM and inter-LLM setups, with PreRR (average Jaccard overlap of predicted label sets) generally lower, showing exact label-level reproduction is harder. They also report a small-scale manual comparison in which the structure of a generated algorithm corresponded step-by-step to reasoning elicited through top-k decoding, suggesting the algorithm is not just a surface story.","pith_inferences":["A natural control experiment the paper does not run would test whether a placebo algorithm (generic steps, or reversed decision rules) reproduces the task outputs just as well; if it does, the reproducibility scores would measure output stability under re-prompting rather than the content of the extracted algorithm.","The inter-LLM reproducibility values could be inflated by the strict 'follow the steps' instruction, which is a strong output-format nudge; stripping the extracted algorithm to its semantic core or shuffling its steps would isolate what content is actually transferred.","Because the intrinsic-reasoning check used a much smaller model (LLaMA 3.2-1B) than the main experiments, the claimed step-wise alignment should be re-tested on the same scale before being treated as evidence of mechanism.","The paper's own PreRR numbers suggest that extracted algorithms are not fully deterministic at the data-point level, so a stronger validation would be checking whether the algorithm's stated conditions correspond to the model's attention or attribution patterns."],"forward_implications":["If ReQuesting works, laypeople in law, medicine, and finance could ask an LLM 'what did you do?' and receive a checkable, step-by-step account of the task, which could raise trust in high-stakes AI-assisted decisions.","Because the extracted algorithms are executed successfully on a different LLM in the inter-LLM setups, the method suggests that task-level knowledge captured by the algorithm is portable across models, not a quirk of one model's prompting.","The consistently high PerRR with lower PreRR implies that generated algorithms faithfully reproduce overall performance but not the exact set of individual predictions, so case-by-case explanations would still require caution.","The manual alignment between the ReQuest algorithm and top-k elicited intrinsic reasoning points toward a possible bridge between behaviorally extracted explanations and mechanism-level accounts, which could become a more rigorous verification procedure."],"supporting_citations":[{"why":"Supplies the statute-prediction task prompt, dataset, and the baseline Macro-F1 that the LLMs are compared against.","marker":"Vats et al., 2023"},{"why":"Provides the ECHR dataset and the human-rights-violation prediction task, plus the fact extraction used in that task.","marker":"Chalkidis et al., 2019"},{"why":"Demonstrates that LLMs can execute natural-language-described algorithms, the premise of the robustness-check prompt.","marker":"Zheng et al., 2024"},{"why":"Shows LLM self-explanations are comparable to traditional explanation methods, lending prior support to using generated explanations.","marker":"Huang et al., 2023a"},{"why":"Chain-of-Thought prompting, the approach that ReQuesting distinguishes itself from by not requiring human-annotated reasoning steps.","marker":"Wei et al., 2022"},{"why":"Provides the top-k decoding-path method used to elicit the model's intrinsic reasoning for the RQ3 alignment check.","marker":"Wang and Zhou, 2024"},{"why":"Defines the Jaccard overlap used in the PreRR prediction-reproduction metric.","marker":"Jaccard, 1912"},{"why":"Supplies the stock-tweet dataset used for the finance stock-movement prediction tasks.","marker":"Xu and Cohen, 2018"},{"why":"Supplies the Reddit Suicide Watch dataset for the mental-health classification task T1.","marker":"Ji et al., 2021a"},{"why":"Supplies the depression-severity Reddit dataset for the mental-health task T2.","marker":"Naseem et al., 2022"}],"fun_headline_variants":["LLMs craft plain-language algorithms that reproduce their own answers","ReQuesting: LLMs write faithful plain-English algorithms for their reasoning","New method makes LLMs explain their decisions in plain steps","LLMs can now explain their own decisions in simple step-by-step language","Ask an LLM to write its steps, then run them to verify its answer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that comparable outputs between the original task prompt and the algorithm-executed prompt mean the algorithm reflects what the model is actually doing; without a control condition, matching outputs could simply reflect the model's tendency to give consistent answers to similarly worded prompts.","fun_headline_variants_meta":{"raw":{"variants":["LLMs craft plain-language algorithms that reproduce their own answers","ReQuesting: LLMs write faithful plain-English algorithms for their reasoning","New method makes LLMs explain their decisions in plain steps","LLMs can now explain their own decisions in simple step-by-step language","Ask an LLM to write its steps, then run them to verify its answer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0007,"raw_usage":{"total_tokens":3142,"prompt_tokens":910,"completion_tokens":2232,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":2140}},"tokens_in":526,"tokens_out":2232,"duration_ms":16218,"temperature":1.0,"reasoning_tokens":2140,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:30:35.445969+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the ReQuest pipeline but replace the extracted algorithm with a placebo or deliberately corrupted one, such as a generic 'classify by your best judgment' instruction or the same steps in shuffled order, and compare the PerRR/PreRR values. If the placebo achieves reproducibility scores comparable to the real algorithm, the metric tracks prompt-recognition rather than the algorithm's content, and the faithfulness claim fails.","supporting_citations":[],"review_version":1}