{"id":"a25136a7-22fb-405d-a999-f39ae33b9d72","arxiv_id":"2607.07467","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"An LLM multi-agent framework (SpaCellAgent) automates end-to-end trajectory inference on single-cell and spatial transcriptomics data, achieving expert-aligned accuracy with 41.2% faster analysis time.","lead":"SpaCellAgent is an LLM-driven multi-agent system that automates trajectory inference analysis on single-cell and spatial transcriptomics data, from data loading through biological report generation. A smart generalist might read it because it promises to replace hours of expert bioinformatics labor with a self-correcting AI pipeline that matches human-quality results.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Benchmark comparison is asymmetric: SpaCellAgent benefits from dynamic parameter optimization while all baselines run with default parameters, making the SOTA claim difficult to attribute to the agent architecture rather than to simple parameter tuning.","rationale":"The reader's verdict of CONDITIONAL is appropriate, but I would reweight the concerns. The reader's primary concern (biological evaluator accuracy) is valid but secondary: the downstream TI metrics in Table 1 provide indirect evidence that the refinement loop produces reasonable results, and an imperfect evaluator would mainly affect convergence speed rather than final output quality. The more load-bearing concern is the asymmetric benchmark: SpaCellAgent's gains over baselines could be largely attributable to parameter optimization rather than the agent architecture itself. This doesn't invalidate the paper's contribution — automated parameter tuning and end-to-end workflow automation are genuinely useful — but it does mean the 'SOTA' framing overstates what has been demonstrated. The paper should either (a) run baselines with tuned parameters to show the agent architecture adds value beyond parameter optimization, or (b) reframe the contribution as 'automated expert-level parameter selection and workflow orchestration' rather than 'SOTA TI performance.' The efficiency claim (41.2%) also has a small sample (5 experts) with subjective success criteria, but this is a lesser concern since the qualitative point — that automation saves time — is well-supported. The paper has real merits: public code, diverse datasets, ablation studies, and cross-model sensitivity analysis. CONDITIONAL remains the right verdict, with the parameter-tuning confound as the primary condition to address.","tokens_in":37129,"tokens_out":2646,"duration_ms":178102,"concrete_test":"Re-run the two strongest baselines (Slingshot and PAGA Tree) on the REAL-GOLD and REAL-SILVER benchmarks with grid-searched hyperparameters (e.g., for Slingshot: clustering resolution ∈ {0.3, 0.5, 0.8, 1.2}, start cluster variants; for PAGA Tree: resolution and neighbor count variants). If the tuned baselines close the gap to within 10% of SpaCellAgent's scores on Corr and F1, the SOTA claim should be revised to 'comparable to expert-tuned baselines while requiring no manual parameter selection.'","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim of SOTA trajectory inference performance rests on Table 1, where SpaCellAgent is compared against DPT, RaceID/StemID, Scorpius, PAGA, PAGA Tree, and Slingshot. §5.1 explicitly states: 'For all baselines, we adopt the default parameter settings as provided in their original implementations.' Meanwhile, SpaCellAgent's executor agent performs dynamic tool selection and parameter configuration via LLM reasoning, and its self-refinement loop iteratively adjusts parameters based on evaluator feedback. This creates a fundamental asymmetry: SpaCellAgent effectively runs one of the same underlying TI algorithms (e.g., PAGA, Slingshot, Monocle3 — all listed in the tool registry, Table 5) but with optimized parameters, while baselines run untuned. The paper attributes the improvement to 'self-refinement and self-evolution mechanisms' (§5.3), but provides no experiment isolating the agent's biological reasoning from the effect of parameter optimization alone. If the performance gap is primarily driven by parameter tuning — which any baseline method would also benefit from — then the contribution narrows from 'agent achieves SOTA' to 'automated parameter tuning outperforms default settings,' which is a substantially weaker claim. The reader identified this as a secondary concern (condition 2), but I consider it more load-bearing than the biological evaluator concern, because Table 1 is the primary quantitative evidence for the headline claim, and the evaluator's accuracy only affects the refinement loop's efficiency, not the fundamental fairness of the comparison. The biological evaluator concern is real but indirect: even an imperfect evaluator could still produce good downstream metrics if the system converges, whereas the parameter-tuning confound directly inflates the SOTA claim.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The manuscript introduces SpaCellAgent, an LLM-driven multi-agent framework for automating end-to-end trajectory inference (TI) analysis on single-cell and spatial transcriptomics data. The system features a planner-executor-evaluator architecture with a self-refinement loop and a cross-task self-evolution mechanism (dual-layer memory). The evaluator agent performs both code validation and biological plausibility checks. The framework is evaluated on synthetic and real-world benchmarks (Saelens et al.) against standard TI baselines (DPT, Slingshot, PAGA, etc.) and in case studies on mouse midbrain and spinal cord injury datasets. The authors report state-of-the-art TI performance and a 41.2% reduction in analysis time compared to human experts.","tokens_in":37350,"tokens_out":1209,"duration_ms":232394,"significance":"The paper addresses a practical bottleneck in computational biology: the fragmentation of TI tools and the need for manual expertise. The multi-agent architecture with a biological evaluator and self-evolution mechanism is a reasonable and novel application of LLM agents to this domain. The provision of verbatim system prompts (Appendix A.2) and a publicly available code repository are notable strengths that aid reproducibility. The evaluation uses established external benchmarks (Saelens et al. [31]), ensuring that the trajectory metrics are not circularly defined.","major_comments":[{"comment":"§5.1 (Baselines): The benchmark comparison in Table 1 is asymmetric. SpaCellAgent benefits from dynamic parameter optimization via its executor and self-refinement loop, while all baselines (DPT, Slingshot, PAGA, etc.) run with default parameters ('For all baselines, we adopt the default parameter settings as provided in their original implementations'). Since SpaCellAgent's tool registry (Table 5) includes PAGA, Slingshot, and Monocle3, the framework effectively runs the same underlying algorithms but with optimized parameters. This makes it difficult to attribute the performance gap to the agent's biological reasoning rather than to parameter tuning. An ablation isolating the agent's reasoning from parameter optimization (e.g., comparing SpaCellAgent against a baseline with grid-search-tuned parameters) is needed to support the SOTA claim.","section":null},{"comment":"§4.2 (Biological evaluator) and §4.3 (Self-refinement): The biological evaluator uses LLM reasoning to flag biologically implausible trajectories, and this assessment underpins the self-correction loop. However, the paper does not benchmark the evaluator's diagnostic accuracy against ground-truth biological assessments. If the LLM-based evaluator systematically misjudges trajectory plausibility, the self-refinement loop could converge on biologically incorrect results that pass the evaluator's checks. A quantitative evaluation of the evaluator's precision/recall on known biological errors would strengthen this load-bearing component.","section":null},{"comment":"§5.3 and Table 2 (Efficiency analysis): The 41.2% efficiency improvement claim is based on a comparison with only 5 human experts. The standard deviations reported (e.g., Step 1: 10.0±1.6 min for humans) suggest moderate variability, but with n=5, the statistical power to establish a robust baseline is limited. The manuscript should acknowledge this limitation or provide a sensitivity analysis to show that the 41.2% figure is stable and not driven by outlier expert performances.","section":null}],"minor_comments":[{"comment":"Table 1: The HIM score for SpaCellAgent on the Real-Silver dataset is 0.322, which is lower than all baselines except RaceID/StemID. This anomaly should be discussed, as it contradicts the 'ranking first in the majority of metrics' claim.","section":null},{"comment":"§5.1 (Model configuration): The temperature for the main agents is set to 0.0, but for self-reflection it is 0.4. The rationale for 0.4 specifically (rather than 0.3 or 0.5) is not justified. A sensitivity analysis over this parameter would be helpful.","section":null},{"comment":"Figure 7: The axis labels and legend are rendered as unicode escape sequences (e.g., /uni00000036/uni00000057...) and are unreadable. This figure needs to be regenerated with proper text rendering.","section":null},{"comment":"Table 3: The dataset identifiers 'S1' through 'S5' are used but not defined in the main text. A brief description of these subsets should be provided for context.","section":null},{"comment":"Appendix B.7: The text references a 'six-task' workflow (Table 11) but then mentions a 'five-step' plan (Table 12). The inconsistency between 5 and 6 tasks/steps should be clarified.","section":null},{"comment":"§5.4 (Case Study): The Mouse SCI dataset is described as 'unpublished' and provided by a collaborating hospital. A brief note on ethical approval or data availability for this dataset would be appropriate, even if it is only available upon request.","section":null}],"recommendation":"major_revision","confidential_remarks":"The core architectural contribution (LLM agents orchestrating TI tools with self-refinement) is sound and interesting. However, the primary quantitative claim of SOTA performance rests on an asymmetric benchmark comparison that confounds agent reasoning with parameter optimization. This is a load-bearing issue that requires an additional ablation or a revised framing of the contribution. The biological evaluator concern is also valid but secondary; it could potentially be addressed with a smaller-scale validation experiment. The paper fits the journal's scope but needs these experimental gaps closed before acceptance."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The three major comments are well-taken. We address each below and describe revisions we will make.","responses":[{"response":"The referee raises a valid concern. We agree that the current comparison in Table 1 does not fully disentangle the contribution of the agent's biological reasoning from the contribution of adaptive parameter tuning. This is a genuine gap in our evaluation. In the revised manuscript, we will add an ablation experiment in which each baseline algorithm (PAGA, Slingshot, PAGA Tree) is run with grid-search-optimized parameters on the same benchmark datasets, and compared against (a) the same algorithm invoked by SpaCellAgent's executor with agent-selected parameters, and (b) the full SpaCellAgent pipeline. This will allow us to isolate the marginal contribution of the agent's reasoning (planner decomposition, biological evaluator feedback, self-refinement) above and beyond parameter optimization. We note that our existing ablation (Figure 7, Table 13) already shows that removing the evaluator or self-evolution degrades task success rate and correlation, which provides partial evidence that the agent architecture contributes beyond mere parameter selection. However, we agree that a direct comparison against grid-search-tuned baselines is the cleaner test and will include it. We will also soften the SOTA claim in the abstract and main text to clarify that the improvement reflects the combined effect of adaptive parameter selection and agent-level reasoning, pending the results of this additional ablation.","revision_made":"yes","referee_comment":"§5.1 (Baselines): Asymmetric comparison — SpaCellAgent benefits from dynamic parameter optimization while baselines run with default parameters. An ablation isolating agent reasoning from parameter optimization is needed."},{"response":"We agree that the biological evaluator is a load-bearing component and that its accuracy should be quantitatively assessed rather than only demonstrated qualitatively through case studies. In the revised manuscript, we will construct a benchmark of trajectory outputs containing known, deliberately injected biological errors (e.g., inverted pseudotime with terminally differentiated cells assigned as root, fragmented manifolds where continuity is expected, biologically impossible lineage transitions such as neurons differentiating into glia without a progenitor intermediate). We will then measure the evaluator's precision and recall in flagging these errors versus correctly passing valid trajectories. We will report these metrics in a new table in Section 5 or the appendix. We acknowledge that constructing a comprehensive ground-truth set of biological errors is non-trivial and inherently incomplete — there exist classes of subtle biological errors that no LLM-based evaluator may reliably catch. We will explicitly state this limitation in the revised text, noting that the evaluator serves as a necessary but not sufficient safeguard, and that final biological validation by domain experts remains essential for novel discoveries.","revision_made":"yes","referee_comment":"§4.2 and §4.3: The biological evaluator's diagnostic accuracy is not benchmarked against ground-truth biological assessments. A quantitative evaluation of precision/recall on known biological errors would strengthen this component."},{"response":"The referee is correct that n=5 is a small sample and limits the statistical robustness of the 41.2% figure. We will address this in two ways in the revision. First, we will add an explicit limitation statement in Section 5.3 acknowledging that the human baseline is based on five experts and that the generalizability of the efficiency claim is therefore constrained. Second, we will provide a sensitivity analysis: we will report the efficiency improvement under leave-one-out exclusion of each expert (i.e., recomputing the baseline mean with each expert removed in turn) to demonstrate that the 41.2% figure is not driven by a single slow or fast outlier. We will present these results in a supplementary table. If any leave-one-out configuration substantially changes the improvement percentage, we will report the range transparently rather than a single point estimate. We will also revise the abstract and conclusion to qualify the claim as 'over 40% improvement in analytical efficiency in our expert benchmark (n=5)' rather than presenting it as a universally established figure.","revision_made":"yes","referee_comment":"§5.3 and Table 2: The 41.2% efficiency improvement is based on only 5 human experts. The manuscript should acknowledge this limitation or provide a sensitivity analysis."}],"tokens_in":36801,"tokens_out":1241,"duration_ms":142399,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Two things to know up front: SpaCellAgent is a competent application of multi-agent LLM architecture to trajectory inference (TI), and the benchmark comparison in Table 1 has a fairness problem that undermines the headline SOTA claim. The stress-test note is right to flag this as the central issue, more load-bearing than the biological evaluator concern the reader emphasized. Here's why: the agent's executor dynamically selects and configures TI tools (PAGA, Slingshot, Monocle3, etc.) with LLM-optimized parameters, while every baseline runs at default settings. So the comparison is effectively 'tuned algorithm vs. untuned algorithm,' not 'agent reasoning vs. static pipeline.' The paper does not include an ablation isolating parameter optimization from agent-level biological reasoning. Without that, the performance gap could be largely attributable to hyperparameter tuning — which any baseline would also benefit from. This doesn't kill the paper, but it does mean the SOTA framing needs revision. The real contribution is 'automated end-to-end TI workflow with self-correction,' not 'agent beats SOTA TI methods.' On the positive side: the code is public, the multi-agent architecture (planner/executor/evaluator/reporter) is well-motivated for this domain, the dual-layer memory and self-evolution mechanism are sensible engineering, and the case studies on mouse midbrain and axolotl regeneration show genuine biological plausibility. The cross-model sensitivity analysis (Table 3) showing consistent performance across DeepSeek-V3, GPT-5.2, and Claude is good evidence the framework isn't overfit to one LLM. The efficiency claim (41.2% time reduction) rests on only 5 human experts with high variance — this is a pilot estimate, not a rigorous benchmark. The biological evaluator's diagnostic accuracy is never independently validated, which matters for the self-refinement loop but is a secondary concern compared to the benchmark asymmetry. Figure 7 has rendering failures that hurt the ablation readability. This paper is for bioinformatics researchers and method developers interested in LLM-driven automation of computational biology workflows. It deserves a serious referee who can push the authors to (1) add a tuned-baseline comparison or at minimum an ablation isolating parameter optimization from agent reasoning, (2) validate the biological evaluator against ground-truth assessments, and (3) scale up the human expert comparison. The core engineering is solid and the application is genuinely useful — the evidence just needs to catch up to the claims.","headline":"Useful LLM-agent automation for trajectory inference, but the SOTA claim is confounded by an asymmetric benchmark setup — baselines run untuned while the agent optimizes parameters.","tokens_in":38317,"tokens_out":591,"would_cite":false,"duration_ms":79015,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"LLM agents autonomously run trajectory inference at expert quality, 41% faster","keywords":["trajectory inference","multi-agent systems","large language models","spatial transcriptomics","single-cell RNA sequencing","self-refinement","computational biology","pseudotime"],"falsifier":"Benchmark the biological evaluator agent's diagnostic accuracy against expert-curated biological assessments across diverse tissue contexts. If the evaluator's false-positive or false-negative rate for biological implausibility is high in novel contexts, the self-refinement loop would be shown to be unreliable.","tokens_in":37162,"feed_emoji":"🧬","tokens_out":1466,"duration_ms":107719,"temperature":0.7,"pith_summary":"The paper proposes SpaCellAgent, a multi-agent system built on large language models that autonomously performs end-to-end trajectory inference (TI) analysis on single-cell and spatial transcriptomics data. The framework assigns distinct roles — a planner that decomposes the biological question into analysis steps, an executor that selects appropriate computational tools and generates runnable code, an evaluator that checks both code correctness and biological plausibility, and a reporter that synthesizes findings into a narrative. Two mechanisms distinguish it from a static pipeline: self-refinement, where the evaluator's feedback drives iterative code correction within a single analysis, and self-evolution, where validated workflows and error-fix pairs are archived in a persistent memory and retrieved for future tasks. The authors claim that this architecture achieves state-of-the-art trajectory reconstruction quality across three benchmark categories (REAL-GOLD, REAL-SILVER, SYNTHETIC) while reducing total analysis time by 41.2% compared to a panel of five domain experts, from 64.6 minutes to 38.0 minutes per analysis. The central object carrying the argument is the biological evaluator agent — an LLM that judges whether inferred trajectories are biologically coherent (e.g., flagging a terminally differentiated cell incorrectly placed at the trajectory root) and routes corrective feedback back to the planner and executor.","feed_headline":"LLM agent team runs trajectory inference at expert quality, 41% faster","feed_subtitle":"Four cooperating LLM agents — planner, executor, evaluator, reporter — autonomously reconstruct cell developmental paths from transcriptomic","key_machinery":"Multi-agent LLM architecture with four specialized roles (planner, executor, evaluator, reporter), coupled with a dual-layer memory (local intra-task and global inter-task) that enables self-refinement within a task and self-evolution across tasks.","core_discovery":"The paper's central claim is that a coordinated team of LLM agents, each handling a distinct phase of trajectory inference analysis, can match or exceed specialized TI algorithms on standard benchmarks while operating autonomously from natural-language input to biological report. The improvement over individual TI methods (DPT, Slingshot, PAGA, etc.) is attributed to the system's ability to dynamically select and configure algorithms per dataset rather than relying on fixed algorithmic assumptions, and to iteratively self-correct both code-level and biology-level errors through the evaluator-driven refinement loop. The self-evolution mechanism further accelerates convergence on new datasets:","pith_inferences":["The system's quality advantage over individual TI algorithms may partly reflect ensemble behavior — by trying multiple algorithms and selecting based on the biological evaluator's judgment, the agent effectively performs a form of model selection that no single baseline can match. If so, the fair comparison might be against a human executing the same multi-algorithm selection strategy, not against","The biological evaluator's reliability is the linchpin of the entire self-correction loop. If the LLM systematically misjudges trajectory plausibility in novel biological contexts — particularly tissues or organisms underrepresented in its training data — the refinement loop could converge on confidently wrong results. A direct benchmark of the evaluator's diagnostic accuracy against expert-curate","The cross-model sensitivity analysis (DeepSeek-V3, GPT-5.2, Claude Sonnet 4) showing consistent performance suggests the gains are driven by the agentic architecture rather than a specific LLM's capabilities, but also implies the system inherits whatever biological knowledge gaps its underlying LLM carries.","The 41.2% efficiency claim is measured against five experts on a standardized six-task workflow. Real-world TI analysis often involves exploratory iteration and dead ends that are not captured by a fixed workflow, so the time savings in practice may differ from the controlled benchmark."],"forward_implications":["If the self-evolution mechanism genuinely accumulates reusable analysis templates, the system's marginal cost per new dataset should decrease over time, potentially making large-scale atlas-level trajectory analysis tractable without proportional growth in human expertise.","The dynamic tool discovery mechanism — where the agent can import and register new packages on the fly — suggests a path toward frameworks that stay current with the rapidly expanding bioinformatics tool ecosystem without manual registry updates.","The biological evaluator's PubMed-augmented fallback for unfamiliar tissue contexts raises the possibility of agent-driven literature synthesis embedded directly into the analysis loop, though its reliability as a ground-truth proxy is unbenchmarked.","If the 41.2% time reduction holds across labs, it could lower the barrier for non-bioinformatician biologists to run expert-quality trajectory analysis, shifting the role of human experts toward validation and hypothesis generation rather than pipeline construction."],"fun_headline_variants":["Multi-agent LLM system automates trajectory inference with 40% efficiency gain","LLM agents self-correct through evaluation loop to match expert trajectory analysis","SpaCellAgent autonomously selects TI algorithms per dataset, cutting manual steps","Four-agent LLM framework reconstructs cell developmental paths from natural language","Self-evolving LLM agents deliver expert-quality trajectory inference end-to-end"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper assumes that the LLM-based biological evaluator — which checks whether inferred trajectories are biologically plausible (e.g., whether the root cell is a progenitor, not a terminally differentiated cell) — provides reliable biological validation. This is load-bearing because the entire self-correction loop depends on the evaluator correctly diagnosing biological errors. The paper validates downstream trajectory metrics against ground truth but does not directly test","fun_headline_variants_meta":{"raw":{"variants":["Multi-agent LLM system automates trajectory inference with 40% efficiency gain","LLM agents self-correct through evaluation loop to match expert trajectory analysis","SpaCellAgent autonomously selects TI algorithms per dataset, cutting manual steps","Four-agent LLM framework reconstructs cell developmental paths from natural language","Self-evolving LLM agents deliver expert-quality trajectory inference end-to-end"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":673,"prompt_tokens":577,"completion_tokens":96,"prompt_tokens_details":null},"tokens_in":577,"tokens_out":96,"duration_ms":61017,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T09:55:23.659053+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Benchmark the biological evaluator agent's diagnostic accuracy against expert-curated biological assessments across diverse tissue contexts. If the evaluator's false-positive or false-negative rate for biological implausibility is high in novel contexts, the self-refinement loop would be shown to be unreliable.","supporting_citations":[],"review_version":1}