{"id":"ba898496-f978-4e6b-a4de-4ff09a00f119","arxiv_id":"2505.00875","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In an agentic task-guidance pipeline, chain-of-thought reasoning did not improve output quality or provide genuine explainability.","lead":"This paper tests whether chain-of-thought reasoning in AI agents helps or explains anything in a multi-step task guidance system. It finds that models producing reasoning text score no better than simpler models, and that the reasoning text often fails to explain the final answer.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The between-model comparison in Sections 5–6 cannot isolate chain-of-thought; DeepSeek-distilled models differ from Llama/Qwen baselines in training, distillation, and decoding, with no within-model CoT toggle.","rationale":"The Reader's weakest_assumption identifies the same load-bearing concern: the DeepSeek versus Llama/Qwen comparison does not isolate chain-of-thought, because model family and distillation procedure vary alongside CoT availability. This is the primary reason the central claim is overstated. The reader's CONDITIONAL verdict is therefore appropriate. I do not see a need to move the verdict: the paper is an early-findings workshop contribution, its qualitative examples are consistent with existing CoT faithfulness critiques, and the recommended fix—a within-model ablation—is a natural extension rather than a reason to reject the paper outright. I would also note, as a secondary concern, that the explainability component lacks a user study; but the confound in the quantitative comparison is the more decisive issue, and the conditional verdict already captures the need for stronger evidence.","tokens_in":12225,"tokens_out":3414,"duration_ms":40947,"concrete_test":"Run a within-model ablation on the same agentic pipeline: for each of Llama-3.1-8B, Qwen-7B, and Qwen-14B, compare standard answer generation against an otherwise identical condition that prompts the same checkpoint to 'think step by step before answering' (or generates a scratchpad then answers), scoring outputs with the same human/LLM rubric used in Section 5. Additionally, where feasible, compare DeepSeek-distilled models' full CoT-conditioned responses against their final-answer-only outputs. If the within-checkpoint CoT manipulation does not reproduce the Fig. 2 gap, the reported difference is attributable to model family or distillation rather than to chain-of-thought itself.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative result (Fig. 2) compares instruction-tuned Llama3-8b and Qwen-7b/14b with their DeepSeek-R1-distilled counterparts. These are not the same checkpoints with chain-of-thought enabled; they are separately RL/distillation-trained models whose outputs include extended reasoning text. Model family, distillation procedure, prompt format, sampling parameters, and the propensity to emit long scratchpads are therefore all confounded with the presence of CoT. Sections 3 and 5 describe scoring by human reviewers and an LLM judge, but no ablation toggles CoT on and off within a fixed checkpoint, and no condition suppresses reasoning tokens from the DeepSeek-distilled models. Consequently, the claim that 'CoT reasoning alone does not lead to better outputs' is not identified by the reported experiment. What is supported is the weaker claim that the reasoning-distilled model family received lower judged answer scores in this agentic pipeline. The explainability half of the central claim is also not directly tested: no end users were asked to use the system, so the assertion that CoT does not improve users' understanding or goal achievement rests on analyst interpretation of qualitative examples rather than on measured user outcomes. The between-model confound is the more load-bearing defect, because it undermines the causal reading of the paper's headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports early findings from an agentic task-guidance pipeline evaluated offline on a participatory dataset of task-based and organizational/social questions. Six LLMs (Llama-3-8B, Qwen-7B/14B, and their DeepSeek-R1-distilled counterparts) are deployed as the LLM in each agent of the pipeline. Answers are scored by human reviewers and GPT-4o as a judge on a -1/0/0.5/1 scale. The paper's headline claims are that chain-of-thought reasoning alone does not lead to better outputs and does not offer explainability. The reported results show higher judged scores for the non-reasoning models, weak correlation between thought scores and answer scores for reasoning models, and a qualitative analysis of three prompt-CoT-output tuples suggesting that CoT can be unhelpful or misleading. The authors acknowledge the preliminary nature of the work and defer ablations to future work.","tokens_in":12583,"tokens_out":5129,"duration_ms":45422,"significance":"If the causal claim were established, the paper would be a useful cautionary contribution to HCXAI, challenging the assumption that surfacing chain-of-thought text in agentic systems provides transparency. The participatory dataset built with manufacturing technicians is a genuine strength, as is the use of both human and LLM-based judges and the authors' explicit acknowledgment of limitations. However, as it stands the central causal claim is not identified by the experimental design, and the explainability claim lacks user evidence. The paper is therefore better read as a preliminary observational study than as a demonstration that CoT has no explanatory value.","major_comments":[{"comment":"The comparison of 'non-reasoning' Llama/Qwen models against DeepSeek-R1-distilled models is confounded by model family, size, distillation procedure, and decoding behavior; there is no within-model ablation that toggles chain-of-thought on and off, and no condition suppresses the reasoning tokens in the DeepSeek models. The paper's own Limitations section states that ablations of each agent are left to future work. Therefore the abstract's claim that 'CoT reasoning alone does not lead to better outputs' is not identified by the reported experiment. What is supported is the weaker claim that the DeepSeek-distilled model family received lower judged answer scores in this agentic pipeline.","section":"§5-6, Fig. 2"},{"comment":"The claim that CoT 'does not improve the ability of end users to better understand systems or achieve their goals' is an empirical claim about users, but no user study is reported. The conclusion rests on the authors' qualitative reading of three tuples (Table 1) and on the weak correlation between thought and answer scores, neither of which measures user comprehension or task success. A user study with a condition that shows or hides CoT text is needed to support the explainability half of the headline claim.","section":"Abstract and §7 Discussion"},{"comment":"The chain-of-thought text shown for the input 'Can you communicate with humans in multiple languages?' is identical to the chain-of-thought shown for tuple B ('How do you ensure data privacy and security?'), yet §7's qualitative analysis of tuple C relies on details (e.g., 'Since the context only lists English') that are not present in the displayed CoT. This suggests either a copy-paste error in the table or an unacknowledged model perseveration; either way, the qualitative evidence for the tuple C discussion as presented is not trustworthy and must be corrected or re-analyzed.","section":"Table 1, tuple C"},{"comment":"The paper reports weak human-LLM judge agreement (Cohen's kappa = 0.48 for org-soc, 0.28 for task) but Figure 2 plots LLM-as-judge scores alongside human scores and uses them in significance tests. Given the weak agreement, the paper should state whether the main effect (reasoning vs. non-reasoning) holds when using only human scores, and should caution against pooling or relying on LLM judge scores for the central comparison.","section":"§6 Results"}],"minor_comments":[{"comment":"There is a typo in the word 'susceptiblity' that should be corrected to 'susceptibility'.","section":"§7 Discussion"},{"comment":"The data counts given as 'N=750/300 (Task), 300 (Org-Soc), 150 (Thoughts)' are unclear; please clarify the exact number of question-answer tuples and thought annotations per condition.","section":"§3 Data"},{"comment":"The caption refers to the 'middle column in the heatmap Figure 2', but the figure appears to be a bar chart; please clarify what the heatmap is and how it relates to the reported bar values.","section":"Fig. 2 caption"},{"comment":"The sentence 'the experts assign scores to every answers generated by the agentic flow using 6 different LLMs' is ambiguous; the six LLMs are the systems being evaluated, while scoring is performed by human reviewers and GPT-4o. Please reword to avoid confusion.","section":"§5 Experiments"},{"comment":"The acronym 'MARIE' appears in the example tuples without prior definition; please define the system name when it is first introduced.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The authors are transparent about the lack of ablations, but the abstract nevertheless states a causal conclusion that the design cannot support. The duplicated CoT in Table 1 needs investigation before the qualitative analysis can be relied upon. If the authors can add a within-model CoT toggle and a small user study, the paper's contribution would be substantially stronger; otherwise the claims should be softened to describe a model-family comparison and an analyst-led interpretive study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: This paper asks the right question—does chain-of-thought give you better outputs or real explainability inside an agentic pipeline?—but its headline claim overreaches its evidence. The comparison is between model families (DeepSeek-distilled reasoning models vs. Llama/Qwen), not CoT on and off within the same model.\n\nWhat's genuinely useful: the authors built a real agentic task-guidance system (perceptors, planners, RAG, safety agent) and evaluated six models on a participatory dataset of task and organizational questions. The result that reasoning-distilled models scored lower on judged answer quality in this pipeline is a concrete, if narrow, data point. The qualitative examples are the best part: they show CoT text drifting into irrelevant machine terminology or committing hasty generalizations, which gives a tangible sense of why raw CoT might not serve as an explanation. The paper also honestly reports weak correlation between thought-quality scores and answer-quality scores, and weak human/LLM-judge agreement.\n\nThe soft spots are significant. The central quantitative claim is not identified: DeepSeek-R1-distilled models are not the same checkpoints with CoT toggled on; they're separately RL-trained models with different decoding behavior. So \"CoT reasoning alone does not lead to better outputs\" is not what the experiment shows. What it shows is that these particular reasoning-distilled models got lower scores here. That's still worth reporting, but the causal reading is confounded. The explainability claim is even less supported—no user study, just the authors' interpretation of three examples. The weak thought-answer correlation is suggestive but doesn't measure whether users understand or achieve goals better.\n\nThe limitations section is honest about dataset size and model per agent, but it doesn't flag the missing CoT ablation, which is the load-bearing issue.\n\nBottom line: this is a workshop-level contribution, useful as a case study and a prompt for a proper follow-up with a within-model CoT toggle and a user study. I'd send it to peer review with a clear expectation of major revision, and I'd want the claims scaled back to what the design can support. I wouldn't cite it as evidence that CoT doesn't work, but I'd cite it as an example of how hard it is to evaluate CoT in an agentic setting.\n\nRecommendation: engage with it, but treat the headline as a hypothesis, not a finding.","headline":"A well-intentioned workshop paper whose evidence is too confounded to support its claim that CoT doesn't help in agentic pipelines—but the qualitative examples and the system itself are worth a look.","tokens_in":12999,"tokens_out":2869,"would_cite":false,"duration_ms":27160,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fails: Chain-of-thought reasoning can't explain or improve LLM outputs","keywords":["chain-of-thought","explainable AI","agentic pipelines","large language models","LLM-as-a-judge","human-centered XAI","task guidance","qualitative content analysis"],"falsifier":"Run a within-model ablation on the same agentic pipeline: take one model family (e.g., Qwen) and compare its responses with chain-of-thought prompting enabled versus disabled on the same benchmark questions. If the CoT-enabled version achieves equal or higher human and judge scores, the paper's claim that CoT alone does not improve outputs would be contradicted.","tokens_in":12037,"feed_emoji":"🤖","tokens_out":7503,"duration_ms":61355,"temperature":0.7,"pith_summary":"This paper sets out to test a common assumption in explainable AI: that chain-of-thought (CoT) text—the step-by-step verbalization some LLMs produce before answering—makes an agentic system's outputs both better and more transparent to end users. Working with a perceptive task-guidance system that helps technicians assemble and disassemble toy vehicles, the authors compared three non-reasoning LLMs with three DeepSeek-distilled reasoning variants that emit CoT. They found that the non-reasoning models received higher human and LLM-judge scores for accuracy, comprehensiveness, and helpfulness on task and organizational questions. A qualitative analysis of the CoT text showed it drifting toward familiar but wrong associations (the Einstellung paradigm) and producing verbose, logically flawed rationalizations that obscure rather than explain. The paper concludes that CoT alone neither improves outputs nor provides explainability; it produces explanations without explainability.","feed_headline":"Fails: Chain-of-thought reasoning can't explain or improve LLM outputs","feed_subtitle":"Non-reasoning models beat CoT-reasoning models on task and organizational questions in an agentic pipeline.","key_machinery":"The central object is the agentic pipeline, a fixed-and-dynamic flow of LLM-based agents (perceptors, planners such as lead/query/answer planners, and actors including the RAG module, question answerer, reformulator, and safety agent) with the same LLM deployed in each agent. Within this pipeline, chain-of-thought is treated as the vehicle of explainability under test: the DeepSeek-distilled reasoning models produce a CoT scratchpad before each response, while the Llama and Qwen models do not. The comparison between these two groups, together with human and LLM-as-a-judge scoring of answers and thoughts, is what carries the argument; the qualitative content analysis of CoT tuples supplies the mechanism by which CoT misleads.","core_discovery":"The paper's central claim is that in an agentic pipeline, chain-of-thought reasoning text does not deliver the two things it is often credited with: better answer quality and explainability. Quantitatively, the DeepSeek-distilled reasoning models, which generate CoT, scored significantly lower than their non-reasoning counterparts on both task-specific and organizational/social questions, as rated by human reviewers and an LLM judge (Wilcoxon rank-sum tests). The reviewer scores of the CoT thoughts were only weakly correlated with the reviewer scores of the answers, suggesting the thoughts do not steer the model to correct responses. Qualitatively, the CoT text exhibited reasoning that fixated on familiar machine-related vocabulary rather than the task context (Einstellung paradigm), made hasty generalizations from incomplete context, and produced excessive text that raised the burden on any user seeking an explanation. The authors argue that such CoT output amounts to explanations without explainability: it does not improve end users' ability to understand the system or achieve their goals.","pith_inferences":["A natural extension beyond this paper is a within-model ablation that toggles chain-of-thought on and off for the same model, which would separate the effect of CoT from differences in model family and distillation.","The 'explanations without explainability' framing likely extends beyond agentic pipelines: any system that emits fluent rationalizations ungrounded in retrieved evidence may mislead users just as much as it informs them.","A testable follow-up would measure end users' actual task performance (time to completion, error rate) with and without CoT text displayed, rather than relying only on expert and judge ratings of answer quality.","The weak correlation between thought scores and answer scores could be exploited as an early-warning signal for unreliable answers, though the paper does not draw this conclusion."],"forward_implications":["Showing users the raw chain-of-thought text of an agentic system will not, by itself, produce actionable transparency; systems need explanations designed for the user's task.","Agentic pipelines that rely on CoT for explainability should be re-examined; the paper's results suggest the traceability of agent calls does not guarantee trustworthy explanations.","Benchmarking reasoning models on real participatory task questions can yield different conclusions than standard reasoning benchmarks, so deployment-oriented evaluation should include human-centered scoring.","If CoT text can actively mislead, safety and accountability mechanisms should not treat CoT as a faithful record of why an answer was produced.","LLM-as-a-judge scores, though weakly correlated with human scores in this study, may still serve as a scalable reference-free indicator for triaging response quality."],"supporting_citations":[{"why":"Supplies the Assembly101 dataset on which the task-based questions are modeled.","marker":"[30]"},{"why":"Defines the DeepSeek distilled reasoning models that generate the chain-of-thought outputs.","marker":"[7]"},{"why":"Provides the Llama3-8b non-reasoning baseline.","marker":"[8]"},{"why":"Provides the Qwen 7b/14b non-reasoning baselines.","marker":"[4]"},{"why":"Introduces chain-of-thought prompting, the technique whose explanatory value is tested.","marker":"[25]"},{"why":"Documents unfaithful chain-of-thought explanations, supporting the paper's critique.","marker":"[36]"},{"why":"GPT-4o serves as the LLM-as-a-judge that scores answers and thoughts.","marker":"[27]"},{"why":"Offers the Einstellung-paradigm account used in the qualitative analysis of CoT errors.","marker":"[23]"}],"fun_headline_variants":["CoT reasoning offers no explainability or performance boost in agentic pipelines","Chain-of-thought: explanations without explainability in agentic systems","CoT doesn't improve answers or help users understand agentic pipelines","In agentic pipelines, CoT thinking adds noise, not clarity or accuracy","Thoughts without thinking: CoT fails to explain or improve in pipelines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the performance gap between the DeepSeek distilled models and the Llama/Qwen models is caused by chain-of-thought, even though the models differ in family, size, and training method and no model was tested both with and without CoT.","fun_headline_variants_meta":{"raw":{"variants":["CoT reasoning offers no explainability or performance boost in agentic pipelines","Chain-of-thought: explanations without explainability in agentic systems","CoT doesn't improve answers or help users understand agentic pipelines","In agentic pipelines, CoT thinking adds noise, not clarity or accuracy","Thoughts without thinking: CoT fails to explain or improve in pipelines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000203,"raw_usage":{"total_tokens":1364,"prompt_tokens":900,"completion_tokens":464,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":383}},"tokens_in":516,"tokens_out":464,"duration_ms":4671,"temperature":1.0,"reasoning_tokens":383,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:31:57.050510+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a within-model ablation on the same agentic pipeline: take one model family (e.g., Qwen) and compare its responses with chain-of-thought prompting enabled versus disabled on the same benchmark questions. If the CoT-enabled version achieves equal or higher human and judge scores, the paper's claim that CoT alone does not improve outputs would be contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Assembly101 dataset on which the task-based questions are modeled."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents unfaithful chain-of-thought explanations, supporting the paper's critique."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GPT-4o serves as the LLM-as-a-judge that scores answers and thoughts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Offers the Einstellung-paradigm account used in the qualitative analysis of CoT errors."}],"review_version":1}