{"id":"b8c94e5f-bb34-4b37-a9f6-30801859adac","arxiv_id":"2610.32114","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A benchmark of 1,858 paired ML experiment designs shows that frontier language models are far from reliable at detecting injected methodological flaws, reaching at best 78% both-orders accuracy and exhibiting strong position bias.","lead":"Tampered ML is a benchmark of 1,858 paired machine learning experiment designs, each pairing an original published experiment with a minimally edited version containing a targeted methodological flaw. It tests whether language models can pick the more sound design, finding that frontier models are far from reliable and highly sensitive to presentation order.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Human validation shows only 66% of items missed by all top models are valid (Table 3), yet these items drive the claim that models are unreliable on hard cases, and the post-stratified estimate of 86% valid rests on small per-stratum samples with single validators.","rationale":"The paper acknowledges the limitation about label quality on hard items (Table 3, §4) and provides a post-stratified validity estimate. However, the load-bearing nature of this assumption is not fully stress-tested: the hardest stratum (0/3 correct) contributes 15.6% of the benchmark but has the lowest validated validity (66%), and the estimate uses single-validator labels. If even half of the invalid/ambiguous items in that stratum were removed, the headline both-orders accuracy would shift non-negligibly. The reader's weakest_assumption flags exactly this issue. The paper's transparent reporting allows a concrete test to resolve it: multi-rater validation of those 79 items. If the result does not materially change the accuracy, the concern is addressed; if it does, the claim should be qualified. I still recommend the ACCEPT verdict because the paper is transparent, the limitation is stated, and the overall 86% validity estimate is a reasonable best effort, but the stress-test reveals a specific empirical vulnerability that merits attention.","tokens_in":26799,"tokens_out":2906,"duration_ms":26679,"concrete_test":"Obtain three independent expert re-judgments (with agreement measured via Fleiss' kappa) for all 79 items in the 0/3-correct stratum from Table 3. Then, for each model, recompute both-orders accuracy on the subset of items that all three validators unanimously deem valid. If the revised accuracy on these consensus-valid items is substantially higher (e.g., >83% for Opus 5) than the reported 77.9%, the central claim that models are unreliable on hard items weakens. Conversely, if accuracy remains essentially unchanged, the concern about label quality in this stratum is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that frontier models are 'far from reliable' depends on their performance on the full benchmark, including the hardest items where all three top models fail. The paper's own human validation (Table 3) finds that in this stratum, only 66% of items are judged valid (52/79); the rest are ambiguous or invalid. Crucially, these 79 reviewed items are the sole basis for the post-stratified estimate of 86% valid overall, and each was reviewed by a single validator with no inter-annotator agreement reported. If a substantial fraction of the 290 items in this stratum (15.6% of the benchmark) are not actually valid—i.e., the tampered design is not consequentially flawed—then model failures on those items do not indicate poor judgment. Removing them would raise models’ both-orders accuracy, potentially from 78% to noticeably higher, undermining the quantitative claim of unreliability. The paper acknowledges the limitation but does not quantify how sensitive the headline accuracy is to label noise in this stratum. Because the benchmark's evaluation set is intentionally filtered to be hard, the hardest items carry disproportionate weight in the conclusion; their label quality is the single most load-bearing assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper introduces Tampered ML, a benchmark of 1,858 item-pairs designed to evaluate whether large language models can distinguish methodologically sound ML experiment proposals from minimally edited alternatives that contain a targeted flaw. Each pair consists of a real experiment from a published paper (original) and a counterfactual version with one injected methodological error. Models must choose the more sound design in both presentation orders. A human validation study on a stratified sample estimates 86% of items are valid (95% CI: 81–91%). Frontier models (e.g., Opus 5 at 77.9% both-orders accuracy) perform above chance but remain far from reliable, show strong position bias, and often reward specificity or transparency even when those features carry the flaw.","tokens_in":27173,"tokens_out":2635,"duration_ms":24571,"significance":"The benchmark targets a core capability for scalable oversight: recognizing methodological flaws in experiment designs. The construction pipeline is carefully designed with multiple automatic gates and human validation, and the paper provides extensive analysis of failure modes (position bias, substituted heuristics, reasoning failures). The finding that frontier models are highly sensitive to presentation order and reward superficial rigor over validity is an important empirical result for the alignment community. The diagnostic variants (flaw disclosure, category tagging) show that many errors are recoverable, pointing toward model improvements. The human validation, while based on single-validator reviews and overrepresenting hard items, is a substantial effort and honestly reported. The benchmark fills a gap between existing error-detection benchmarks (FLAWS, PaperAudit) and agent-focused benchmarks (MLE-bench).","major_comments":[{"comment":"The central claim that models are 'far from reliable' rests on their performance on the full benchmark, including the hardest stratum where all three top models fail (0/3 correct, 290 items, 15.6% of the benchmark). Human validation in this stratum found only 66% of reviewed items valid (52/79), compared to 93% in easier strata. The paper reports a post-stratified estimate of 86% overall valid, but does not quantify how the headline both-orders accuracy would change if items in the 0/3 stratum were excluded or reweighted based on per-stratum validity rates. Since roughly a third of reviewed items in this stratum may not be genuinely flawed, model failures on those items do not indicate poor judgment. I request a sensitivity analysis: report both-orders accuracy for Opus 5 and the next best models after (a) excluding all 0/3-stratum items, (b) excluding items in that stratum that are flagged as likely invalid under conservative assumptions, and (c) a bootstrap that respects per-stratum validity uncertainty. Without this, the quantitative precision of the unreliability claim is unclear.","section":"Table 3, Section 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The post-stratified estimate of 86% valid assumes that, within each baseline-model stratum, the reviewed items are representative of the leaderboard population. The 0/3 stratum has only 79 reviewed items out of 290 (27% coverage), and Appendix E notes that items were selected to match validator expertise. Selection on expertise could correlate with validity if reviewers tend to receive items from their subfield, where they may be stricter or more lenient. The paper should discuss this selection mechanism and whether it introduces bias into the post-stratification. A simple robustness check would be to reweight the per-stratum estimates using only the random-sampled portion of the validation set (if any) or to present a range of estimates under different non-response assumptions.","section":"Section 4, Table 3, Appendix E","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The leaderboard items are a subset of the candidate pairs that passed the automatic gates, further filtered to remove items that a subset of models solve in both orders (the difficulty filter). The paper states this is not a validity filter, but it means the reported accuracies are on a harder-than-average slice. The central claim that frontier models are 'far from reliable' may be less concerning if the full candidate set (which includes easier items) yields higher accuracy. The paper should report the total number of candidate pairs before the difficulty filter (if available) and the accuracy of at least one top model on the unfiltered set, or discuss how the difficulty filter affects the generality of the conclusion. If the filter removes many easy items, the headline accuracy could be lower than what would be observed in a realistic deployment setting where flaws vary in subtlety.","section":"Section 2 (Difficulty Filter), Section 3","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The caption for Figure 1 refers to 'Fable 5' but the model is introduced only later in the text. Provide a brief clarification in the caption or a footnote early in the text that Fable 5 is an Anthropic model.","section":"Abstract, Section 3, Figure 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The right column reports stratum shares (e.g., 'c1 > c2 33.5%'). The caption says the c2 > c1 stratum is omitted; consider adding a note about what that omitted stratum contains (presumably very few items) to prevent confusion.","section":"Section 3, Figure 6","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The drop statistics on the right of Figure 9 are in percentage points but labeled as 'drop' without units. Explicitly label as 'pp drop' or 'percentage point drop'.","section":"Section 3.5, Figure 9","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The footnote says 'Not valid combines ambiguous and invalid human verdicts.' It would be helpful to also show the count of ambiguous vs. invalid separately in the table or a supplementary table, since they have different implications for benchmark quality.","section":"Section 4, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The table title says 'Blind agreement predicts validity' but the association is descriptive, not causal. Consider rephrasing to 'Association between blind-stage-1 choice and final validity'.","section":"Appendix E, Table 9","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Several references to model system cards (e.g., Anthropic 2026, OpenAI 2026a, 2026b) do not have stable URLs or DOIs in the bibliography. For reproducibility, provide URLs or permanent identifiers where available.","section":"General","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"minor_revision","confidential_remarks":"The paper is well-executed and the benchmark is a valuable contribution. My main concern is the sensitivity of the headline result to label noise in the hardest items, but the authors address most limitations honestly. The recommended sensitivity analysis should be straightforward. The paper is a good fit for the journal and should be published after the minor revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This is a solid piece of work that does what a good benchmark should: define a clean task, build a controlled dataset, run careful evaluations, and report limitations in proportion. The core idea—take published experiments, inject one targeted flaw, ask models to pick the better design in both orders—is simple and effective. The pipeline is well thought out, with multiple gates to ensure faithfulness and non-trivial difficulty. The results are informative: even Opus 5 tops out at 78% both-orders accuracy, models show strong position bias, and they tend to reward specificity or conventional practice over actual validity. The diagnostic variants are a nice touch, showing that telling models there is a flaw recovers many errors.\n\nThe authors are unusually transparent about limitations. They flag the LLM-in-the-loop generation, the single-validator human audit, and the overrepresentation of hard items in the validation sample. They report that only 66% of items all three top models miss are judged valid (Table 3). This is the main soft spot the stress test zeroes in on, and it is a real one. If a non-trivial fraction of the hardest items have debatable or non-consequential flaws, the headline accuracy could be higher than reported. The post-stratified estimate of 86% valid overall helps, but the per-stratum sample is small and relies on single validators. I would have liked a sensitivity analysis showing how accuracy changes under worst-case assumptions about label noise in the 0/3 stratum. That said, the qualitative findings (position bias, failure modes) are robust even if a few items are mislabelled, and the authors do not overclaim—they call the benchmark diagnostic, not a certification.\n\nMinor point: the paper uses LLMs to generate and gate items, which could introduce systematic blind spots. The authors acknowledge this. The fluke test and pointwise scoring are nice extras that strengthen the case.\n\nThis paper deserves a serious referee. It will be cited by anyone working on AI evaluation for scientific reasoning. I would bring it to my reading group.","headline":"A careful, transparent benchmark for testing whether models can spot methodological flaws in ML experiments. The central finding holds up, though the human validation on the hardest items has the load-bearing soft spot the authors themselves flag.","tokens_in":27628,"tokens_out":1763,"would_cite":true,"duration_ms":22606,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Tampered ML  benchmark shows frontier LLMs cannot reliably detect flawed ML experiment designs, achieving at most 78% accuracy on controlled pairwise comparisons.","keywords":["LLM evaluation","methodological soundness","experiment design","benchmark","scalable oversight","flaw detection","AI alignment"],"falsifier":"If a single frontier model (e.g., a future version of Opus or GPT) achieved >95% both-orders accuracy on the full 1,858-item set while demonstrating robust qualitative reasoning (e.g., consistently identifying the specific causal mechanism of the flaw across all flaw categories), the central claim that models are 'far from reliable' would be falsified. Alternatively, if an adversarial human study showed that the majority of items that all strong models fail are actually invalid (the paper's validation suggests ~34% may be invalid in that stratum), the error rate attributed to model judgment would be partially explained by label noise.","tokens_in":28330,"feed_emoji":"🔬","tokens_out":2095,"duration_ms":20008,"temperature":0.7,"pith_summary":"This paper introduces Tampered ML, a benchmark of 1,858 paired experiment proposals where one version is a faithful original from a published ML paper and the other is minimally edited to introduce a single methodological flaw. The central question is whether language models can identify which of the two designs is more methodologically sound. The paper finds that even the strongest frontier models (Opus 5, Fable 5) reach only about 78–79% both-orders accuracy, remain far from reliable, and are highly sensitive to the order in which proposals are presented. Models often reward specificity, transparency, or conventional practice even when those very features carry the injected flaw. The results imply that current LLMs are not trustworthy for automated scientific oversight, even in a simplified setting where the comparison is tightly controlled.","feed_headline":"Frontier LLMs flunk experiments on ML experimental judgment","feed_subtitle":"Even the best model, Opus 5, misses flaws in 22% of controlled pairwise comparisons—and is easily swayed by presentation order.","key_machinery":"The Tampered ML benchmark: a pipeline that takes published ML experiments, extracts structured facts, rewrites them as self-contained proposals, then generates minimally edited counterfactuals that inject a single methodological flaw from an 8-category taxonomy (e.g., confounded comparison, blocked mechanism, measurement mismatch). Each item consists of a research context, original proposal, tampered proposal, and held-out flaw description. Models must select the more methodologically sound design in both presentation orders, enabling both per-order and both-orders accuracy metrics.","core_discovery":"The paper establishes that frontier LLMs, despite performing above chance, are not reliable judges of methodological soundness in ML experiment designs. In a controlled pairwise setting where one proposal is faithful to a published experiment and the other contains a single targeted flaw, the best model (Opus 5) achieves 77.9% both-orders accuracy (credit only if correct in both presentation orders). Models show strong position bias: they often select the original only when presented first. Qualitative analysis reveals that models substitute heuristics—treating specificity, transparency, or conventional practice as evidence of validity—even when these features are precisely what makes the tampered design flawed. Diagnostic interventions (flaw disclosure, category hints, review decomposition) recover many errors, indicating the models often recognize the flaw but fail to weight it decisively.","pith_inferences":["The benchmark's construction relies on LLMs to generate and validate items, which introduces a circularity risk: if the generator models share blind spots with the evaluated models, the benchmark may underrepresent flaws that all current LLMs miss. The 86% human validation rate partially mitigates this but does not eliminate it.","The position bias and confidence shifts suggest that models' reasoning chains are fragile and context-dependent; this has implications for interpretability: even when models produce correct rationales, the reasoning process may be inconsistent.","If future models are trained on this benchmark (as warned by the authors), they may learn to detect the specific tamper patterns rather than general methodological reasoning. The benchmark should be used diagnostically, not as a training set for improving automated review.","The failure mode where models accept a tampered design because it 'explicitly aligns with common practice' points to a deeper limitation: current LLMs lack a causal understanding of experimental validity and substitute surface-level norms."],"forward_implications":["Current LLMs cannot be trusted to autonomously review or filter experiment proposals in alignment research or other scientific domains, as they miss even explicitly injected single flaws.","Best-of-N selection and generate-and-filter pipelines that rely on LLM judgment will pass flawed designs and reject sound ones, potentially amplifying methodological errors.","The finding that models reward specificity and conventional practice over genuine validity suggests that automated oversight may entrench standard but suboptimal methodologies.","Diagnostic scaffolds (disclosing the flaw category) substantially improve performance, indicating that the benchmark measures not only flaw detection but also the ability to prioritize among multiple methodological concerns.","The gap between pairwise and pointwise scoring (4–19 percentage points lower) implies that real-world performance, where models evaluate a single proposal, is even worse than the pairwise leaderboard suggests."],"supporting_citations":[{"why":"Provides context on AI systems being used to conduct AI research (the motivating application for evaluating experimental judgment).","marker":"Lu et al. (2024)"},{"why":"Discusses scalable oversight, the broader research area where automated judgment of experiment designs is relevant.","marker":"Bowman et al. (2022)"},{"why":"Related benchmark that inserts errors into scientific papers, which the current work complements by focusing on paired experiment designs rather than full-paper audit.","marker":"Xi et al. (2025) (FLAWS)"},{"why":"Formalizes the generator–discriminator gap, which the paper invokes to distinguish between generating sound experiments and judging them.","marker":"Saunders et al. (2022)"},{"why":"Provides the framing on test-time compute scaling that the paper uses to analyze the impact of reasoning effort on accuracy.","marker":"Snell et al. (2025)"}],"fun_headline_variants":["Best LLM misses 22% of flaws in ML experiments","Frontier LLMs prefer flashy but flawed designs","Model judgment sways with presentation order","LLMs mistake specificity for soundness in ML"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The ground-truth labels are reliable: the injected flaws are real and consequential, and the original proposals are indeed more methodologically sound. The human validation, while supportive (86% validity estimated), is based on a single validator per item and overrepresents hard items, so if many unvalidated items have debatable flaws, the conclusion that models are unreliable would be overstated.","fun_headline_variants_meta":{"raw":{"variants":["Best LLM misses 22% of flaws in ML experiments","Frontier LLMs prefer flashy but flawed designs","Model judgment sways with presentation order","LLMs mistake specificity for soundness in ML"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000316,"raw_usage":{"cost":0.000315588,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000315588,"upstream_inference_prompt_cost":9.3548e-05,"upstream_inference_completions_cost":0.00022204},"total_tokens":1666,"prompt_tokens":873,"completion_tokens":793,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":731}},"tokens_in":873,"tokens_out":793,"cost_source":"openrouter_usage","duration_ms":7864,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":731,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-26T13:16:33.687933+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"falsifier":"If a single frontier model (e.g., a future version of Opus or GPT) achieved >95% both-orders accuracy on the full 1,858-item set while demonstrating robust qualitative reasoning (e.g., consistently identifying the specific causal mechanism of the flaw across all flaw categories), the central claim that models are 'far from reliable' would be falsified. Alternatively, if an adversarial human study showed that the majority of items that all strong models fail are actually invalid (the paper's validation suggests ~34% may be invalid in that stratum), the error rate attributed to model judgment would be partially explained by label noise.","supporting_citations":[{"cited_title":"Self-critiquing models for assisting human evaluators","cited_arxiv_id":"2206.05802","evidence_quote":"Formalizes the generator–discriminator gap, which the paper invokes to distinguish between generating sound experiments and judging them."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the framing on test-time compute scaling that the paper uses to analyze the impact of reasoning effort on accuracy."}],"review_version":1}