{"id":"d3eb7c68-f79d-40c0-b333-c2d4935bba8e","arxiv_id":"2607.28841","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A local 7B-parameter model and four-agent orchestration system automate neuroimaging workflows, improving held-out tool-use accuracy from 40% to 69% and cutting token use by roughly 89% versus NeuroClaw.","lead":"CyberNeuro is a desktop workbench that uses a small local AI model to run neuroimaging analysis pipelines from plain-language requests while keeping patient data on-site. It reports large reductions in manual effort, model tokens, and user actions compared with a prior agentic system, plus zero observed outbound traffic in tested conditions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Held-out QC-62 may be contaminated by adversarial-hardening examples; the 69.4% accuracy is not yet credible.","rationale":"The paper's central quantitative claim—that WandaMind raises held-out domain accuracy from 40% to 69%—is supported only by the QC-62 evaluation. The reader identifies contamination of QC-62 as the weakest assumption, and this is indeed the most load-bearing point. The iterative training process in §3.7.3 is explicitly failure-driven: Round 2 diagnoses 'four diagnostic cases' where escalation rates dropped, and Round 3 adds 21 'Hardening Counterexamples.' If any of these examples were drawn from or semantically similar to QC-62, the evaluation is circular. The stated decontamination (Jaccard ≥ 0.55 token overlap) is too weak to catch paraphrase or semantically equivalent cases, and the paper does not state whether the later-round examples were even subjected to it. This concern directly threatens the validity of the 69.4% figure, which underpins the abstract and the system's claimed advantage. The abstract/body mismatch (NeuroBench vs. QC-62) is a real reporting problem but secondary: even if corrected, the contamination issue would remain. A concrete audit—releasing training and test sets and checking semantic overlap—would settle the matter. Without such evidence, the paper should remain conditional, not accepted at face value.","tokens_in":20700,"tokens_out":5799,"duration_ms":61772,"concrete_test":"Release the QC-62 prompts and the complete training corpus (including all Round 2 augmentation and the 21 Round 3 hardening examples). Compute pairwise semantic similarity (e.g., sentence-transformer embedding cosine or GPT-4 paraphrase judgment) between every training/hardening example and every QC-62 case. If any pair exceeds a conservative threshold (e.g., cosine > 0.85) or is judged a paraphrase, retrain without those examples and re-evaluate on QC-62. Additionally, require the authors to state the provenance of the four diagnostic cases in §3.7.3; if they came from QC-62, the reported 69.4% must be recomputed under a properly separated protocol.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline '40% to 69%' rests entirely on the held-out QC-62 evaluation (§4.1, Table 3). The paper describes three training rounds (§3.7.3, Fig. 2c): Round 2 'ACT-Only Augmentation' and Round 3 'Adversarial Hardening' with '+21 Hardening Counterexamples'. The decontamination procedure in §3.7.1 (token-level Jaccard ≥ 0.55 against QC-62) is stated for the 'core pool of 722 training trajectories'; it is never stated that the Round 3 counterexamples, or the 'four diagnostic cases' that motivated Round 2, were decontaminated against QC-62, nor that QC-62 was not used as the diagnostic set for failure mining. The 29.1-point gain is concentrated in the exact categories these rounds targeted (Quality Judgment 0/11→7/11, Command Construction 0/6→4/6, Escalation 0/4→4/4). If any hardening/diagnostic examples were derived from QC-62 cases—even paraphrases below the Jaccard threshold—the improvement is an artifact of test-set contamination, not domain adaptation. The paper provides no evidence that QC-62 remained truly unseen during iterative model development, so the central claim is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CyberNeuro, a Windows desktop agentic workbench for neuroimaging and clinical data analysis that keeps all model inference, tool execution, and data within a local privacy boundary. Four agents (Planner, Validator, Dispatcher, Reporter) mediate between a natural-language interface and a pinned skill library via a Model Context Protocol bridge. The core technical contribution is WandaMind, a QLoRA-adapted Qwen2.5-7B model trained with response-only supervision, token-level decontamination against a held-out QC-62 set, and three rounds of iterative failure mining and adversarial hardening. The paper reports that WandaMind improves held-out tool-use accuracy from 40.3% to 69.4%, reduces DICOM-to-BIDS human interventions from 24 to 0 across ten cohorts, and consumes about 89% fewer tokens per NIfTI than NeuroClaw. It also reports post-install network denial tests and claims zero outbound traffic in the disconnected condition. The authors are explicit that the 27B-comparator advantage is not significant and that per-NIfTI normalization favors NeuroClaw because of different output counts.","tokens_in":21073,"tokens_out":6894,"duration_ms":67277,"significance":"If the claims hold, the paper is a useful systems contribution: it demonstrates that a small locally deployed model can be hardened to route and refuse within a bounded neuroimaging tool space, and that deterministic validation gates can make unattended execution safe enough for cohort-scale conversion. The strengths are the explicit separation of LLM planning from deterministic contract, policy, and artifact validation; the provenance receipts; the reproduction of the 10-cohort comparison under two backends; and the candid treatment of limitations (host-wide netstat not process-attributed, per-NIfTI normalization favoring the baseline, adaptation being decision hardening rather than general capability gain). However, the central 69.4% held-out claim is not yet credible because the provenance of the Round 2/3 hardening examples relative to QC-62 is not disclosed, and the abstract overstates the evaluation as being on the public NeuroBench suite. The token-efficiency headline is also stated inconsistently in the abstract, though the body is more careful.","major_comments":[{"comment":"Section 3.7.3 describes three training rounds, including Round 2 'ACT-Only Augmentation' motivated by 'four diagnostic cases' and Round 3 'Adversarial Hardening' with '+21 Hardening Counterexamples', but it never states that these examples were passed through the Jaccard decontamination against QC-62 described in §3.7.1 (which is phrased only for the 'core pool of 722 training trajectories'). The 29.1-point held-out gain in Table 3 is concentrated in exactly the categories those rounds targeted (Quality Judgment 0/11→7/11, Command Construction 0/6→4/6, Escalation 0/4→4/4, Fig. 2d). If any diagnostic or hardening example was derived from QC-62 cases, even as a paraphrase below the Jaccard threshold, the claimed held-out accuracy is inflated. The paper provides no evidence that QC-62 remained unseen during iterative development. Please disclose the provenance of the 4 diagnostic cases and","section":"§3.7.3, §4.1 (Table 3)"},{"comment":"The Abstract and Conclusion state that the held-out accuracy improvement (40%→69%) is demonstrated 'on the public NeuroBench suite,' but the model evaluation in §4.1 is described only on 'QC-62, a held-out collection of 62 neuroimaging tool-use and quality-control cases,' and the system evaluation in §4.2 uses a custom 10-cohort suite. No statement in the paper explains how QC-62 relates to the public NeuroBench suite, and no NeuroBench leaderboard or external benchmark results are provided. As written, the headline claim is not verifiable from the manuscript; either report the exact relationship to NeuroBench or change the abstract and conclusion to say 'a held-out internal benchmark (QC-62)'.","section":"Abstract, §4.1, §8"},{"comment":"The token-efficiency headline is inconsistent across the paper. The Abstract says 'aggregate token count of about 10.6%' compared to NeuroClaw, but Tables 4 and 6 give total tokens 68,775 vs 2,001,492, i.e., 3.4% of NeuroClaw's aggregate tokens; 10.6% is the per-NIfTI ratio (1,127.5/10,646.2). Section 5.2 itself concedes that per-NIfTI normalization favors NeuroClaw because NeuroClaw emits 188 volumes while CyberNeuro triages to 61. The manuscript should report total-token ratios as primary and per-NIfTI ratios as secondary, and correct the abstract.","section":"Abstract, §4.2, §5.2"}],"minor_comments":[{"comment":"Each total row contains an extra unlabeled numeric value (33.72, 32.58, 13.65) after 'Time (s)'; presumably seconds per NIfTI, but it is not defined and does not match the table header. Please add a column label or remove the value.","section":"Tables 4–6"},{"comment":"The figure states '9.3×' but the table values give 10,646.2/1,127.5 ≈ 9.4. The '89.4% reduction' wording in §4.2 is consistent with 10.6% of baseline, so this is a local arithmetic inconsistency.","section":"Figure 4b"},{"comment":"Per-category counts (e.g., 11 quality-judgment cases, 6 command-construction cases, 4 escalation cases) are reported in Fig. 2d but not in Table 3. Include the n per category in the table so the abstention-recall and category-level percentages are interpretable.","section":"Table 3 / Fig. 2d"},{"comment":"The 'four diagnostic cases' that motivated Round 2 are not described. At minimum, specify whether they come from training data, a development set, or from the QC-62 test set itself; this is directly relevant to the contamination concern in my first major comment.","section":"§3.7.3"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The decisive issue is QC-62 contamination. If the authors can confirm that the 4 diagnostic cases and 21 hardening counterexamples were generated without access to QC-62, or if they perform an external holdout evaluation, the paper becomes a solid systems contribution. If those examples were derived from QC-62 failures, the 69.4% result and the paper's central claim collapse. I therefore recommend major revision, not rejection, because the issue is addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this is a real systems paper, not a hype job. The authors built a local multi-agent neuroimaging workbench (CyberNeuro) with a fine-tuned 7B model (WandaMind) and evaluated it end-to-end on ten DICOM-to-BIDS cohorts and on a held-out QC-62 tool-use benchmark. The best part is the results section, which is unusually honest: it says outright that the model gains come from abstention behavior, not from better tool selection (tool selection is flat; argument validity is actually worse than base), that the 27B comparator's edge over the 7B base is not significant, that per-NIfTI token normalization favors the baseline, and that they claim no computational speedup. The failure-injection test for artifact validation is a nice touch. That kind of candor earns credit.\n\nThe main thing to flag is the QC-62 contamination question. The abstract says 'public NeuroBench suite' but the body evaluates on a custom QC-62 set, and the conclusion repeats the NeuroBench claim — that mismatch needs fixing. More importantly, the decontamination description in §3.7.1 seems to cover the core training pool, but §3.7.3's Round 2 diagnostic cases and Round 3's '+21 Hardening Counterexamples' are never explicitly said to have been filtered against QC-62. The gains are concentrated precisely in the categories those rounds targeted (quality judgment, command construction, escalation). It may well be fine — the Jaccard filter could apply to all records — but the paper leaves it ambiguous, and that ambiguity is load-bearing for the headline 69.4% number. This is the first question I'd ask a referee to chase.\n\nAlso, I could not verify the claimed code/data release from the text; the URL appears only in the abstract, and the reader's report says no artifacts were found. For a systems paper legitimately claiming FAIR4RS compliance, that's a real gap.\n\nNet: the architecture and system-level evaluation look solid, and the authors' own caveats in Sections 4.4 and 5 are more careful than most. The soft spot is narrow but real: the held-out model evaluation needs a written guarantee that the iterative hardening rounds never touched QC-62, plus a correction of the NeuroBench terminology. If they add that, the 69.4% claim becomes credible; until then, treat it as unverified but plausible.\n\nFor your purposes: worth reading if you work on agentic scientific assistants or privacy-preserving neuroimaging tooling. I'd send it to peer review with a request for that clarification, and I'd want the artifacts before citing the accuracy number.","headline":"Serious systems paper with an honest results section, but the 40→69 held-out accuracy claim needs a clearer decontamination statement before it can be trusted.","tokens_in":21549,"tokens_out":2281,"would_cite":true,"duration_ms":28908,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A local 7B model, tuning only ~1% of its weights, drives the neuroimaging workflows that currently need cloud agents — held-out accuracy jumps from 40% to 69%, and data never leaves the machine.","keywords":["agentic neuroimaging","local language model","privacy-preserving computing","DICOM-to-BIDS conversion","safe abstention","QLoRA adaptation","multi-agent orchestration","brain imaging data structure (BIDS)"],"falsifier":"Release the full training corpus — all three rounds, including the 21 hardening counterexamples and the act-only augmentations — and run both token-level and semantic (embedding-similarity) decontamination against the 62 QC-62 prompts. Any semantic near-duplicate beyond the stated Jaccard threshold would inflate the 69.4% figure, while a clean audit would vindicate it. Independently, a prospective replication on unseen scanners and sites, demanding the same zero-intervention, BIDS-validator-clean outcome, would test whether the architecture rather than the specific test set carries the result.","tokens_in":20657,"feed_emoji":"🧠","tokens_out":25645,"duration_ms":222564,"temperature":0.7,"pith_summary":"The paper's central claim is that clinical neuroimaging workflows — converting raw DICOM scans into the standard BIDS layout, running processing pipelines, and quality-checking outputs — can be automated end to end by an agentic system whose entire language model runs on the researcher's own workstation, with no data leaving the machine. Its specialist 7B model, WandaMind, lifts accuracy on a held-out 62-case suite from 40.3% to 69.4%, and the paper shows the gain is not better tool-following but safe abstention: false tool calls drop from 100% to 0%. At the system level, ten heterogeneous cohorts are converted to BIDS-validator-clean datasets with zero human interventions — versus 24 for a cloud-based comparison agent — and about 89% fewer tokens per converted volume. Under process-level network denial, all tested tasks complete with zero sent bytes, supporting the local-first privacy claim. The consequence, if the results hold: laboratories bound by data-use agreements and without cloud budgets can bring agentic automation to protected imaging data on commodity hardware.","feed_headline":"Local 7B model lifts MRI-agent accuracy from 40% to 69%","feed_subtitle":"A ~89% token saving over cloud agents, zero human interventions, and no data leaving the workstation.","key_machinery":"WandaMind's decision boundary, enforced by a deterministic validation layer, carries the argument. WandaMind is a 7B base adapted with response-only QLoRA (parameter-efficient fine-tuning on quantized weights; 1.06% of parameters trainable) in three rounds — supervised initialization, act-only augmentation, adversarial hardening — on a corpus decontaminated against QC-62 by token-level Jaccard similarity. The orchestration layer (Planner, Validator, Dispatcher, Reporter) keeps the LLM routing-only: nothing executes until validation passes, and a run succeeds only when artifact checks confirm a genuine derivative — without this, all four malformed cases exited 0 and were reported complete. Ov","core_discovery":"A small local language model can orchestrate real neuroimaging analysis if orchestration is 'LLM-routed but tool-deterministic': the model proposes, deterministic layers dispose. WandaMind, a 7B base specialized via response-only QLoRA on ~700 decontaminated trajectories plus adversarial hardening, scores 69.4% on held-out QC-62 versus 40.3% (base) and 48.4% (27B); the entire margin is decision quality (abstain recall 0→100%, false actions 100→0%), tool selection unchanged. End to end, ten cohorts convert to validator-clean BIDS datasets in one request each, with 0 interventions and 1,127.5 tokens per volume versus 10,646.2 for the comparison agent; network denial shows zero sent bytes. The","pith_inferences":["Scale may be the wrong lever for bounded tool orchestration: the evidence puts decision calibration and deterministic enforcement ahead of model size, implying that reliable agents in constrained tool domains might be bought with small specialized models plus validators rather than with larger base models.","The routing-only contract — LLM proposes, validator disposes, success requires an inspectable artifact — is data-agnostic, so a direct extension would port it to other protected biomedical pipelines, such as pathology-slide triage or genomic variant curation, wherever a deterministic checker can verify the derivative.","The paper's stated long-context weakness suggests a hybrid it leaves implicit: keep the local 7B as the default router and escalate genuinely hard planning to a stronger model inside the same validation envelope, using the measured token savings as the budget for such escalation."],"forward_implications":["Labs restricted by IRB and data-use agreements can run agentic automation without transmitting protected imaging data: the default configuration is fully local, and process-level network denial showed zero sent bytes for all tested tasks.","DICOM-to-BIDS conversion becomes schedulable unattended: zero human interventions across ten cohorts, versus 24 for the comparison agent — the paper states this as a capability boundary, not a margin.","Token economics favor local orchestration: about one-ninth the tokens per NIfTI volume with the local model, and a roughly 29x gap on total tokens per cohort because the local system triages away non-anatomical series instead of converting everything.","The success criterion is what makes unattended execution trustworthy: a run counts as successful only if the expected scientific derivative exists and passes output checks — the failure-injection test shows that without artifact validation, all four malformed cases were reported complete with exit code 0.","In bounded tool-use domains, a specialized 7B model can outperform a 27B model because the larger model acts on half of the cases it should refuse; the binding constraint is calibrated abstention, not raw capability."],"fun_headline_variants":["Local 7B model beats cloud agents in MRI analysis with 89% token savings","Privacy-first agentic MRI workbench cuts token use by 89%","Agentic LLM does MRI work with 0 human interventions","69% accuracy with local model: MRI agents go privacy-first"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the held-out QC-62 cases were never used, even indirectly, to shape WandaMind's behavior: the paper's decontamination (§3.7.1) removes training candidates only by token-level Jaccard similarity ≥ 0.55, which cannot catch semantic near-duplicates, so if any adversarial-hardening counterexample or act-only augmentation was derived from a QC-62 case, the 69.4% accuracy is inflated.","fun_headline_variants_meta":{"raw":{"variants":["Local 7B model beats cloud agents in MRI analysis with 89% token savings","Privacy-first agentic MRI workbench cuts token use by 89%","Agentic LLM does MRI work with 0 human interventions","69% accuracy with local model: MRI agents go privacy-first"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000687,"raw_usage":{"total_tokens":3010,"prompt_tokens":859,"completion_tokens":2151,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":2085}},"tokens_in":603,"tokens_out":2151,"duration_ms":14435,"temperature":1.0,"reasoning_tokens":2085,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T01:30:57.247284+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Release the full training corpus — all three rounds, including the 21 hardening counterexamples and the act-only augmentations — and run both token-level and semantic (embedding-similarity) decontamination against the 62 QC-62 prompts. Any semantic near-duplicate beyond the stated Jaccard threshold would inflate the 69.4% figure, while a clean audit would vindicate it. Independently, a prospective replication on unseen scanners and sites, demanding the same zero-intervention, BIDS-validator-clean outcome, would test whether the architecture rather than the specific test set carries the result.","supporting_citations":[],"review_version":1}