{"id":"537867ce-9ef0-42fa-93e1-33e8bd1fb45c","arxiv_id":"2507.17303","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A single pathology vision-language model trained with supervised and reinforcement fine-tuning beats four prior MLLMs across 72 ROI-level and slide-level tasks.","lead":"SmartPath-R1 is a multimodal AI system for pathology that handles many tasks on cell images and whole-slide images, and its training method encourages step-by-step reasoning without needing human-written explanations. It reports large gains over four existing medical vision-language models on 72 benchmark tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"WSI-level claims rest on unverified preprocessing: with M=1024 tokens and Table 44 counts far exceeding available slides, 'WSI-level' inputs are likely patches, not whole slides.","rationale":"The reader's weakest assumption is exactly the concern I identified: the WSI-level results depend on inputs that are never specified as true whole slides, and the reported sample counts are inconsistent with whole-slide datasets. The paper's strongest claim is not just that SmartPath-R1 is accurate on 72 tasks, but that it is the only compared model handling both ROI- and WSI-level tasks. That claim fails if the 'WSI-level' tasks are evaluated on patches. The tables and statistical tests appear internally consistent, so this is not a numerical error; it is a construct-validity gap in the central claim. The proposed concrete test would settle the concern by inspecting the released data loader and test manifests, or by running the model on true WSIs. Since the reader already issued a CONDITIONAL verdict based on this issue, my stress-test does not change the recommended outcome.","tokens_in":43707,"tokens_out":4518,"duration_ms":48647,"concrete_test":"After the promised code release, check the data loader and test manifest for the WSI-level tasks: for every row in Table 44, record the original file extension/path and the image size after T^{M,P}. If the manifest contains pre-extracted patch files (e.g., 512x512 PNGs) rather than whole-slide .svs/.tif paths, the WSI claim is invalid. Quantitative follow-up: take the 191 TCGA-LUAD evaluation samples, run SmartPath-R1 with M=1024 on the actual stored inputs and on full-resolution WSIs transformed by T^{M,P}; if accuracy on true WSIs is substantially below the reported 0.860 or fails, the benchmark is patch-level, not WSI-level.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition for the central claim ('simultaneously addresses WSI-level tasks') is that the stimuli called WSI-level in Tables 34-43 are actually whole-slide images or a faithful whole-slide representation. That condition is not established. Section 4.2 and 4.6 set the visual token budget M=1024 for WSI-level inputs and describe only a deterministic resize T^{M,P}; no tiling, aggregation, multi-instance pooling, or slide-level fusion is described anywhere in the methods. With P=28, 1024 tokens bound the input to roughly 896x896 pixels, orders of magnitude below diagnostic whole-slide dimensions. Table 44 reports 18,390 training samples for TCGA-LUAD (and 38,427 for TCGA-BRCA, 30,789 for TCGA-LGG), while TCGA contains only hundreds of diagnostic WSIs per project. These counts force the samples to be patches or heavily tiled crops labeled by slide, not whole slides. Evaluation sets like 191 LUAD samples could be either 191 WSIs or 191 crops; the paper does not say. If the 26 WSI-level tasks are actually patch-level inputs with slide-level labels, the average rank of 1.1 on those tasks (Figure 2d) is not evidence of whole-slide diagnostic ability, and the uniqueness claim in Figure 2a ('only SmartPath-R1 handles WSI') loses its advertised meaning. This is the single most load-bearing weakness; all other issues (baselines, code release) are secondary.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SmartPath-R1, a pathology multimodal large language model (MLLM) built on Qwen2.5-VL with a mixture-of-experts LoRA architecture, trained in two stages: scale-dependent supervised fine-tuning and task-aware GRPO with reward functions for classification, VQA, detection, and segmentation. The authors curate a large dataset of about 2.3M ROI samples and 188K WSI samples and evaluate on 72 tasks spanning ROI classification, detection, segmentation, VQA and WSI classification/VQA, reporting an average rank of 1.1 (first place in 68 tasks) against Qwen2.5-VL, LLaVA-Med, Quilt-LLaVA, and PathoR1, with SmartPath-SFT also appearing in most tables. The central claim is that a single MLLM can simultaneously address both ROI-level and WSI-level tasks while producing chain-of-thought-style diagnostic reasoning without explicit CoT supervision.","tokens_in":43988,"tokens_out":7415,"duration_ms":73400,"significance":"If the WSI-level evidence were properly established, this would be a notable step toward a unified pathology co-pilot, with a large curated training corpus and evaluation across many public datasets. Strengths include the use of public baseline models, explicit reward functions that avoid chain-of-thought annotation, and systematic reporting of confidence intervals and significance tests. However, the load-bearing WSI-level claim currently rests on an unverified and likely patch-based input protocol, and the absence of code/checkpoints and of specialist detection/segmentation baselines tempers the reproducibility and clinical significance of the comparison. The ROI-level results appear internally consistent and are the most defensible contribution at this stage.","major_comments":[{"comment":"The description of WSI-level inputs is inconsistent with whole-slide processing. With patch size P=28 and maximum token count M=1024, the transformation T^{M,P} bounds an input to roughly 896x896 pixels, while diagnostic WSIs are gigapixel images. Table 44 reports 18,390 training samples for TCGA-LUAD and 38,427 for TCGA-BRCA, numbers that far exceed the number of diagnostic slides available in those TCGA projects; these samples must therefore be patches or tiles labeled with slide-level labels. No tiling, aggregation, downsampling, or multi-instance pooling pipeline is described anywhere in the methods. Consequently, the WSI-level results in Tables 34-43 and Figure 2d do not demonstrate whole-slide-image understanding, and the uniqueness claim in Figure 2a ('only SmartPath-R1 handles WSI') is overstated as presented. The authors must either provide the actual WSI processing pipeline, including any slide-level fusion mechanism, or explicitly re-scope the claims to patch-level inputs with slide-level labels.","section":"Sections 4.2, 4.6; Table 44"},{"comment":"There is a potential slide-level data leakage for the WSI-level tasks. The training and evaluation sample counts for the same TCGA project (e.g., TCGA-LUAD: 18,390 training samples vs. 191 evaluation samples; TCGA-BRCA: 38,427 vs. 452) strongly suggest that patches from the same slides could have been used in both training and test, because the number of samples exceeds the number of available whole-slide images. The paper states that overlapping samples were screened and removed, but it does not specify whether the split was performed at the slide or patient level. If the split is not slide-disjoint, the reported accuracies on WSI-level tasks are likely inflated by patch-level leakage. Please clarify the split protocol and, if necessary, re-evaluate with slide-disjoint training and test sets.","section":"Section 4.3; Table 44"},{"comment":"There is an inconsistency in the number of WSI-level VQA tasks. Section 2 states 17 WSI-level VQA tasks, while the subsection 'WSI-Level VQA' states 13; the sum 8+14+14+10+9+17 = 72 matches the stated total of 72 tasks, but if the 13-task count is used the total becomes 68. The tables suggest 13 internal SlideInstruction tasks plus 4 BCNB tasks, which would reconcile the counts, but this should be stated explicitly. Additionally, Figure 2a marks the four baseline models as not handling WSI tasks (x entries for WSI CLS and WSI VQA), yet Tables 34-43 report their performance on those tasks; the figure caption should clarify that all models were evaluated on all tasks and that the check marks denote training coverage or native capability, not the set of evaluated tasks.","section":"Section 2; Section 'WSI-Level VQA'; Table 39"},{"comment":"The detection and segmentation comparisons are mostly against other MLLMs plus the single foundation model BiomedParse; no specialized detectors (e.g., Mask R-CNN) or segmenters (e.g., U-Net, nnU-Net) are included. Since SmartPath-R1's segmentation masks are produced by MedSAM from the model's predicted bounding boxes, the segmentation comparison measures the quality of box proposals under a fixed promptable segmenter rather than end-to-end segmentation ability. To support the wording in Section 2 that SmartPath-R1 achieves 'state-of-the-art performance' in detection and segmentation, the authors should either include specialist baselines or temper the claim to 'state-of-the-art among evaluated MLLMs and BiomedParse.'","section":"Sections 4.4-4.5; Figures 4-5"}],"minor_comments":[{"comment":"The CCRCC dataset is modified from its original 6-class version to a 4-class version by excluding the background and heterogeneous-tissue classes. This post-hoc class filtering changes the task definition and should be disclosed prominently, with a discussion of its effect on comparability with previously published CCRCC results.","section":"Section 4.3 (CCRCC)"},{"comment":"Several confidence intervals appear internally inconsistent; for example, the TCGA-COAD row for SmartPath-R1 reports F1 = 0.977 with 95% CI (0.577, 0.940), where the upper bound is below the point estimate. Please verify all bootstrap or parametric CI computations across the extended tables.","section":"Table 35"},{"comment":"The Code Availability statement says the code 'will be released' at a GitHub URL, but no code or model checkpoints are currently provided. For a methods-heavy paper, this limits reproducibility of the central results; please provide a working repository or state access conditions.","section":"Code Availability"},{"comment":"The ranking is described as being over five models (rank 1 to 5), but most tables include SmartPath-SFT as a sixth method. Please clarify whether the average rank of 1.1 is computed over the five named baselines or over all six methods, and report the rank of SmartPath-SFT in the latter case.","section":"Section 2 and Figures 2-8"},{"comment":"There are minor typographical inconsistencies, such as 'BRCAS' in Table 44 versus 'BRACS' elsewhere, and the WSI-level sample counts in Table 44 are labeled 'samples' without specifying whether they are whole-slide images, patches, or tiles, which is directly related to the major comment on WSI-level preprocessing.","section":"Table 44 and text"}],"recommendation":"major_revision","confidential_remarks":"The WSI-level issue is the main correctness risk in this submission. The paper would be considerably stronger if reframed as a multimodal patch-level model with slide-level supervision, or if it presented a genuine whole-slide evaluation with a described tiling/aggregation pipeline and slide-disjoint splits. The ROI-level results alone are substantial and could support a revised submission after the WSI claims are re-scoped or properly validated. The absence of code and checkpoints is also a serious reproducibility gap for a methods paper at this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: SmartPath-R1 is a real step forward for ROI-level pathology MLLMs – a single model doing classification, detection, segmentation, VQA with reasoning traces, and the average-rank results against other MLLMs look solid. The second thing: the WSI-level half of the paper does not support the whole-slide claim. With M=1024 visual tokens and P=28, the input is at most roughly 896x896 pixels, and Table 44 lists 18,390 training samples for TCGA-LUAD, which is far more than the number of WSIs in TCGA. These are tiles or crops labeled by slide, and no tiling or aggregation method is described. So \"WSI-level classification/VQA\" in Tables 34-43 should be read as patch-level classification with slide-derived labels. That is a load-bearing overclaim: the uniqueness in Figure 2a depends on WSI handling.\n\nWhat is genuinely new: combining scale-dependent SFT, task-aware GRPO rewards for detection/segmentation/VQA, and LoRA-based MoE routing. The individual pieces are known, but the assembly covering six task families in one model is new relative to PathoR1's ROI-VQA-only scope. The curation is also substantial – 2.3M ROI and 188K \"WSI\" samples – and the comparisons across 72 tasks are internally consistent, with SmartPath-R1 ahead of Qwen2.5VL, LLaVA-Med, Quilt-LLaVA, and PathoR1 in almost every table.\n\nSoft spots, in order. First, the WSI issue above. It is fixable if the authors clarify the preprocessing, provide tiling details, and run a true slide-level baseline (e.g., multiple instance learning or a representative tile aggregation). Second, detection and segmentation are only compared against BiomedParse, a general medical foundation model, not against specialized pathology detectors/segmenters like Mask R-CNN or HoVer-Net. That weakens the \"versatile\" claim somewhat but does not invalidate the comparison against other MLLMs. Third, code and checkpoints are promised but not released; for a systems paper this matters.\n\nMinor: the CCRCC class filtering is disclosed in Section 4.3, so it's not hidden, but a reader might want it in the abstract-level table. No sign of circularity: the rewards are computed on training data, and the test sets are independent.\n\nWho this is for: anyone working on pathology MLLMs or multimodal medical AI. It deserves a serious referee, but the WSI-level claims need to be verified or revised before the paper's central narrative is accepted. If I were editor, I'd send it to review and flag the WSI preprocessing as the main question for the authors.","headline":"A strong ROI-level pathology MLLM paper that overstates its WSI-level capability; the WSI results are almost certainly on tiles, not whole slides.","tokens_in":44627,"tokens_out":2796,"would_cite":false,"duration_ms":29497,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper presents SmartPath-R1, a single multimodal large language model that simultaneously handles ROI-level classification, detection, segmentation, and VQA along with whole-slide classification and VQA, reporting top average rank…","keywords":["multimodal large language model","computational pathology","whole-slide image analysis","reinforcement learning","group relative policy optimization","mixture of experts","visual question answering","diagnostic reasoning"],"falsifier":"Run SmartPath-R1 on genuine gigapixel whole-slide images with a documented tiling, aggregation, or downsampling pipeline and compare accuracy against the patch-level numbers; if accuracy collapses to tile-level performance or the model cannot process a real slide under the 1024-token cap, the WSI-level claim is not supported.","tokens_in":43471,"feed_emoji":"🔬","tokens_out":4515,"duration_ms":48081,"temperature":0.7,"pith_summary":"The paper tries to establish that one multimodal large language model can act as a pathology co-pilot across both region-of-interest and whole-slide tasks, combining classification, detection, segmentation, and visual question answering in a single system. It claims that diagnostic reasoning can be learned through reinforcement rewards rather than expensive chain-of-thought annotations, and that the resulting model outperforms existing pathology and general vision-language models on a broad benchmark. A sympathetic reading takes these results as evidence that a unified, reasoning-enhanced model is a viable path toward practical AI assistance in pathology.","feed_headline":"One pathology AI model ranks first on 68 of 72 tasks","feed_subtitle":"SmartPath-R1 handles detection, segmentation, and slide-level questions in one system.","key_machinery":"The load-bearing mechanism is a two-stage training pipeline built on a Qwen2.5-VL base. Scale-dependent supervised fine-tuning adjusts image resolution and visual token budget per task, using 256 tokens for ROI-level tasks and 1024 tokens for WSI-level tasks. Task-aware GRPO then optimizes task-specific rewards for classification, detection, segmentation, and VQA, with a format reward enforcing structured <think></think> and <answer></answer> outputs; a mixture-of-experts LoRA mechanism routes different task families to specialized adapters, and segmentation masks are produced by feeding predicted boxes to the pre-trained MedSAM model.","core_discovery":"The central discovery is that a single MLLM, trained with scale-dependent supervised fine-tuning followed by task-aware group relative policy optimization, can simultaneously solve ROI-level classification, detection, segmentation, VQA, and WSI-level classification and VQA. On 72 tasks, the model achieves an average rank of 1.1 and ranks first on 68 of them, while the strongest prior pathology reasoning model achieves an average rank of 2.9 and first place on 3 tasks. The paper further claims this is the only compared model that handles all six task types at once, and that the reinforcement stage improves reasoning without requiring procedural annotations.","pith_inferences":["Editorial inference: the same two-stage recipe of scale-dependent supervised fine-tuning plus task-aware GRPO could transfer to other gigapixel medical imaging settings, since none of the components is pathology-specific, but the paper does not demonstrate this.","Editorial inference: if the WSI evaluation inputs are actually tiles labeled by slide rather than true whole-slide images, the practical route to genuine WSI diagnosis would require a tiling-and-aggregation wrapper around the model, not the raw 1024-token input described.","Editorial inference: the average-rank headline rewards breadth across many tasks; a deployment-focused study would need to weigh per-task accuracy and failure modes, which the paper does not provide."],"forward_implications":["A single trained model can cover the main pathology task families, so a diagnostic assistant no longer needs separate systems for ROI analysis, detection, segmentation, and whole-slide interpretation.","Pathological reasoning can be elicited through reinforcement rewards instead of chain-of-thought annotations, lowering the cost of building explainable pathology models.","The task-aware reward design produces structured reasoning traces that make model outputs more inspectable and potentially easier for clinicians to verify.","The model's strong results on external validation sets suggest it can generalize to pathology datasets outside its training distribution."],"supporting_citations":[{"why":"Supplies the base architecture and pretrained weights that SmartPath-R1 is built on.","marker":"[25]"},{"why":"Provides the strongest prior pathology reasoning MLLM, used as the main comparative baseline.","marker":"[27]"},{"why":"Supplies the GRPO reinforcement learning algorithm used for task-aware fine-tuning.","marker":"[33]"},{"why":"Provides the pre-trained MedSAM model that converts predicted bounding boxes into segmentation masks.","marker":"[29]"},{"why":"Supplies the PathCap and PathInstruct image-text and instruction datasets used in supervised fine-tuning.","marker":"[24]"},{"why":"Supplies SlideInstruction-Caption and SlideInstruction-VQA, the main WSI-level training and evaluation data.","marker":"[57]"},{"why":"Supplies Quilt-1M, a large corpus of histopathology image-text pairs used in fine-tuning.","marker":"[2]"},{"why":"Supplies PathVQA, an ROI-level visual question answering dataset used for training and evaluation.","marker":"[56]"}],"fun_headline_variants":["Pathology AI takes first place in 68 of 72 tasks","One pathology model wins 68 of 72 benchmarks","Pathology co-pilot tops 68 of 72 tasks","All six pathology tasks in one AI, with 68 wins","Reasoning pathology AI wins 68 of 72 tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the inputs labeled whole-slide images carry true whole-slide diagnostic context; the reported training-sample counts suggest they may be tiles labeled by slide, and no tiling or aggregation pipeline is described.","fun_headline_variants_meta":{"raw":{"variants":["Pathology AI takes first place in 68 of 72 tasks","One pathology model wins 68 of 72 benchmarks","Pathology co-pilot tops 68 of 72 tasks","All six pathology tasks in one AI, with 68 wins","Reasoning pathology AI wins 68 of 72 tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000786,"raw_usage":{"total_tokens":3468,"prompt_tokens":944,"completion_tokens":2524,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":2440}},"tokens_in":560,"tokens_out":2524,"duration_ms":19872,"temperature":1.0,"reasoning_tokens":2440,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:51:13.639141+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SmartPath-R1 on genuine gigapixel whole-slide images with a documented tiling, aggregation, or downsampling pipeline and compare accuracy against the patch-level numbers; if accuracy collapses to tile-level performance or the model cannot process a real slide under the 1024-token cap, the WSI-level claim is not supported.","supporting_citations":[{"cited_title":"InProceedings of the Computer Vision and Pattern Recognition Conference, 5134–5143 (2025)","cited_arxiv_id":null,"evidence_quote":"Supplies SlideInstruction-Caption and SlideInstruction-VQA, the main WSI-level training and evaluation data."},{"cited_title":"neural information processing systems 36, 37995–38017 (2023)","cited_arxiv_id":null,"evidence_quote":"Supplies Quilt-1M, a large corpus of histopathology image-text pairs used in fine-tuning."}],"review_version":1}