{"id":"fdd95a26-e86a-4780-9300-0eaa6a26ffd2","arxiv_id":"2607.23631","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A learnable sequential token router with noise-gated Soft Top-K training and Hard Top-K inference cuts WSI visual context ~36.6× while holding 74% SlideBench accuracy on frozen SlideChat.","lead":"PathSelect learns which tissue patches matter in gigapixel pathology slides by sequential routing, then keeps only ~45 tokens for a frozen vision-language model. It nearly matches a dense baseline at 36× fewer visual tokens, which could make slide-level AI practical on ordinary GPUs.","discovery_kind":"new_method","skeptic_critique":null,"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper reformulates token pruning for gigapixel whole-slide images as a text-conditioned autoregressive selection process. A pointer-decoder Scorer sequentially selects patch tokens with a learned stop action, enabling per-sample adaptive budgets. Because hard top-K selection is non-differentiable, training proceeds through a continuous surrogate: alive-weighted, geometrically decayed pointer scores (Eq. 10) are polarized by a Soft Top-K operator (Eq. 11), then used in a variance-preserving Gaussian noise gate (Eq. 12) whose output is repaired by a diagonal-attention Denoiser (Eq. 13) before the frozen SlideChat LLM. At inference, the gate and Denoiser are detached and a deterministic Hard Top-K policy on the Scorer alone selects tokens. On SlideBench (TCGA) the method reports 74.00% overall accuracy with an empirical average of ~45 tokens under a K=128 cap, versus 74.81% for the uncompressed SlideChat upper bound at ~1642 average tokens (a claimed 36.6× reduction), plus zero-shot results on SlideBench (BCNB) and WSI-VQA*. Ablations cover the Denoiser mask, score aggregation, length penalty λ, and budget K.","tokens_in":13064,"tokens_out":3249,"duration_ms":78617,"significance":"If the numbers hold, this is a useful contribution to a real bottleneck: WSI token counts make dense VLM ingestion expensive, and a frozen-backbone plugin that recovers ~99% of the uncompressed model's accuracy at ~3% of the tokens, trained single-stage on one GPU, is a practically attractive recipe. Specific strengths worth crediting: (i) the framework makes per-sample token counts data-dependent via a learned stop action rather than a fixed budget, which is the right formulation for slides of heterogeneous complexity; (ii) the ablation matrix (Table 2a/b, Table 3) is broader than typical, including mean±std over 2–3 runs for the λ sweep; (iii) zero-shot transfer to BCNB and WSI-VQA* exceeds the uncompressed SlideChat baseline (56.39 vs 54.14; 60.76 vs 60.18), which is a non-trivial falsifiable signal that selection is not merely overfitting TCGA. The train/inference decoupling via a VP noise gate is a reasonable surrogate-gradient design, and the diagonal-attention ablation gives it at least indirect empirical support.","major_comments":[{"comment":"Table 1 vs Table 3 inconsistency in the headline numbers. Table 1 reports 74.00% overall with 'Pruning Rate=97.27%, LP=2', and the abstract and §4.3 cite an empirical average of 44.86 tokens. Table 3's LP=2.0, K=128 row — nominally the identical configuration — reports 73.80±0.29% average and 45.84 average tokens, and Table 2 again quotes 74.00% for the full model. Either Table 1 is a different run/cherry-picked seed than the mean reported in Table 3, or the configurations differ in an unstated way. Since 74.00% vs 74.81% (SlideChat upper bound) is the paper's central 'near-lossless' claim, the authors must reconcile which number is the mean over runs and report Table 1 with the same mean±std protocol as Table 3.","section":"§4.2 Table 1 / §4.4 Table 3"},{"comment":"The claim of 'consistently outperforming sampling-based counterparts' (abstract, §4.2) is not supported by the baseline matrix. Table 1 contains no token-reduction baseline applied to the same SlideChat backbone — no uniform/random sampling of SlideChat to K=128, and no training-free pruner (VisionZip [45], ToMe [11], attention-magnitude ranking, all cited) plugged into the identical frozen base at a matched token budget. As written, the only same-backbone comparison is the uncompressed upper bound, and the remaining baselines are different VLMs wholesale (LLaVA-Med, Quilt-LLaVA, MedDr, GPT-4o), so the gains cannot be attributed to the selection mechanism rather than the SlideChat base. At minimum: SlideChat + random-K and SlideChat + a training-free scorer at matched average token counts are needed to isolate the contribution of the learned sequential policy. This is the load-bearing ab","section":"§4.1 Baselines / Table 1"},{"comment":"The efficiency claims are unquantified where it matters. §4.3 and the abstract assert 'exceptionally low sequential token selection latency' and 'significantly accelerating downstream generative processing', and Table 1 reports FLOPs (1.76T vs 133.3T). But the method's selection loop is itself autoregressive — up to K=128 serial pointer-decode steps over N+1 memory slots — so wall-clock latency is not obviously favorable, and no latency, throughput, or memory measurements are reported anywhere. Figure 3 shows only token counts. At least one timing column (selection latency, end-to-end prefill+decode time per slide, vs SlideChat) is required to substantiate the practical-efficiency contribution; FLOPs alone do not capture the serial selection overhead.","section":"§4.3 Efficiency Analysis / Table 1"},{"comment":"The train–inference mismatch is argued but not directly measured, and one ablation is conspicuously absent. Table 2 ablates the Denoiser mask and the score aggregation, but there is no ablation removing the VP noise gate / Denoiser pathway entirely (e.g., STE-only training feeding the soft-weighted features directly, or training with hard top-K + REINFORCE-style estimation). Since the entire methodological contribution rests on the VP-gate-plus-diagonal-Denoiser surrogate being faithful to the hard-drop deployment path (Eqs. 10–13), the paper should show that this machinery outperforms the simpler differentiable alternatives it replaces — otherwise the added complexity is unjustified. Relatedly, the Soft Top-K target k = running batch average Kb (Eq. 11) makes the training polarization target batch-composition-dependent; its stability should at least be commented on.","section":"§3.3–3.4 / Table 2"},{"comment":"The Clinical-subset improvement over the upper bound (+3.06%, 74.49% vs 71.43%) is presented as evidence that selection 'enhances' reasoning (§4.2), but Table 3 shows the Clinical column has std up to ±1.44 across runs, and Table 1 appears to be single-run. A +3.06 point gain over the uncompressed model is within noise at this reported variance and should be hedged or supported with significance testing; as stated it invites over-interpretation of a single seed.","section":"§4.2 bullet on Clinical Reasoning"}],"minor_comments":[{"comment":"Figure 4's claim of accuracy stability 'even in extreme high-sparsity scenarios where only 4 to 5 tokens are preserved' is not verifiable as presented: no per-bin sample counts or numeric values are given. Please annotate bin counts; stability in sparsely populated tail bins may be an artifact.","section":"§4.3 / Figure 4"},{"comment":"Hyperparameters are under-specified for reproduction: values of β, τ, T_max, the g_txt annealing schedule, Scorer/Denoiser depth and width, and training epochs/optimizer settings do not appear in §4.1. No code release is mentioned. Please add an implementation-details table and state whether code will be released.","section":"§4.1"},{"comment":"Terminology: the text repeatedly calls the selection 'sequential token selection' with 'exceptionally low sequential token selection latency', but never defines the per-step cost relative to the N-sized memory; complexity of the pointer loop (O(T·(N+1)·D)) should be stated explicitly alongside the LLM prefill savings.","section":"§3.2 / §4.3"},{"comment":"Eq. (15): K is described as 'determined dynamically by the initial activation of the stop action' and the loss is said to backpropagate through the alive trajectory; please make explicit how the discrete count K enters the differentiable length penalty (presumably via Σ_t A_t), since as written the penalty term's gradient path is ambiguous.","section":"§3.4, Eq. (15)"},{"comment":"Several typos and formatting issues: double period after Eq. (10) reference in §4.3; 'LLaV A-Med' spacing in Table 1; 'a inflated budget' in §2.1 should be 'an inflated budget'; 'Pain Point 1/2/3' labels are introduced but only 1 and 3 are numbered in §2.1, with 2 appearing in §2.2 — consider consolidating.","section":"§2, §4.3, Table 1"},{"comment":"The UMAP visualization (Figure 5) is suggestive but the interpretive claim — that marginal-occupancy of selected tokens reflects 'complementary pathological outliers' — is post hoc. Either soften the language or provide a quantitative counterpart (e.g., pathologist-annotated ROI overlap of selected patches, which would substantially strengthen the clinical-relevance argument).","section":"§4.3 / Figure 5"},{"comment":"SlideBench (BCNB) and WSI-VQA* are described only as zero-shot; please state dataset sizes and whether the 'Random Baseline' row in Table 1 is chance-level accuracy for each benchmark, to help readers calibrate the ~24–25% values.","section":"§4.1 / Table 1"}],"recommendation":"major_revision","confidential_remarks":"The reference list contains a dense cluster of self-citations from the same group on closely related token-pruning preprints ([8], [12]–[15], [21], [32], [39]), several appearing to be arXiv versions of a shared lineage leading to this submission. The manuscript does not clearly delineate what is new here relative to [15] ('Learnable token sparsification for efficient gigapixel whole slide image reasoning', same first author and venue overlap) and [12]–[14], which appear to introduce the noise-gating and stepwise-selection machinery this paper builds on. I recommend the editor ask the authors for an explicit novelty statement against these four works; if [15] is essentially the same system, the incremental contribution may be narrower than the abstract suggests. This is also the likely explanation for the missing pruning-baseline comparisons on the SlideChat backbone — worth pressing on in revision regardless."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful bit is straightforward: they plug a sequential pointer scorer between frozen SlideChat encoder and LLM, train it with variance-preserving noise gating plus a diagonal-only denoiser so Soft Top-K stays differentiable, then throw the training machinery away and run Hard Top-K / stop-token at inference. Under K=128 they land at 44.86 tokens on average (~36.6× vs the dense ~1642-token baseline) and 74.00% overall on SlideBench-TCGA, essentially matching the uncompressed 74.81% upper bound and beating the compute-matched medical VLMs. Clinical even edges the dense model. That is a real systems win for single-GPU WSI VLM work.\n\nWhat is actually new is not “token pruning” or “pointer decoding” in the abstract—those are established—but the specific decoupled training path: alive-weighted geometric scores → Soft Top-K polarization → VP noise mix → diagonal attention denoise to block leakage, plus the stop token and length penalty that let K become data-dependent. Ablations in Tables 2–3 point the right way (diagonal beats global; full alive×β beats last-step/uniform; LP and budget sweeps show the expected sparsity–accuracy trade-off, with a couple of multi-run stds). UMAP and trajectory figures are consistent with the redundancy story they tell. Citation pattern is normal for the niche; self-cites to related pruning preprints are present but not load-bearing.\n\nSoft spots, in proportion: main headline numbers lack error bars (only the LP sweep has ± over 2–3 runs); no code or released checkpoints; and the central assumption—that optimizing the continuous noisy path produces a scorer whose discrete Hard Top-K policy is faithful at test time—is asserted rather than measured with a direct train–infer mismatch diagnostic. Free parameters (λ, β, τ, running-k, text gate) are ordinary. None of that breaks the claim; it just means the result is still one careful re-implementation away from being fully trusted.\n\nThis is for people building or deploying pathology MLLMs who care about context length and single-GPU reality. It deserves a serious referee. I would engage: read the method section, try the plugin idea on our stack, and cite the efficiency numbers if they hold.","headline":"Practical frozen-backbone plugin that gets near-SlideChat accuracy on ~45 tokens via sequential pointer + VP noise/diagonal denoise; solid applied result with the usual train–infer surrogate gap.","tokens_in":13933,"tokens_out":571,"would_cite":true,"duration_ms":11903,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"PathSelect turns whole-slide token pruning into a learned sequential choice, keeping ~45 patches on average and matching full-context diagnostic accuracy.","keywords":["Gigapixel Whole-Slide Images","Sequential Token Selection","Token Pruning","Vision-Language Models","computational pathology","Soft Top-K","variance-preserving noise gate"],"falsifier":"Measure the train–inference selection mismatch directly: if the set of patches retained by Hard Top-K at test time diverges sharply from the soft alive-weighted scores that received gradient, or if accuracy collapses when the diagonal denoiser mask is replaced by full attention, the claimed transfer fails.","tokens_in":14118,"feed_emoji":"🔬","tokens_out":844,"duration_ms":21441,"temperature":0.7,"pith_summary":"Gigapixel pathology slides produce so many visual tokens that vision-language models cannot afford to read them all. Static sampling and training-free pruning often throw away sparse but decisive tumor cues. This paper reframes pruning as an autoregressive routing problem: a lightweight scorer walks the slide, picks complementary diagnostic regions one by one, and stops when enough evidence is in hand. During training a variance-preserving noise gate and a diagonal-attention denoiser make the discrete choices differentiable while freezing the pretrained slide encoder and language model; at inference only the scorer and a hard top-k remain. The result is an empirical average of 44.86 tokens under a 128-token cap, roughly 36.6 times fewer than the dense baseline, with 74 percent overall accuracy on SlideBench (TCGA) and competitive zero-shot transfer.","feed_headline":"45 patches beat 1,600 for whole-slide diagnosis","feed_subtitle":"A learned stop-and-select router cuts pathology tokens ~37× while matching full-context accuracy","key_machinery":"PathSelect: an alive-weighted Soft Top-K score drives a variance-preserving noise gate that mixes each patch with isotropic Gaussian noise; a diagonal-attention Denoiser restores the representation without letting unselected tokens leak semantics; at inference the gate and denoiser are removed and a deterministic Hard Top-K plus learned stop token execute adaptive trajectory termination.","core_discovery":"The authors show that a decoupled sequential selection plugin, trained end-to-end with continuous noise gating and then discarded at test time, can reduce whole-slide visual context to an average of 44.86 tokens while recovering essentially the full diagnostic accuracy of the uncompressed SlideChat baseline and beating compute-matched sampling and biomedical VLMs.","pith_inferences":["The same noise-gate-plus-stop-token pattern could transfer to other long-context multimodal settings (long video, multi-page documents) where discrete selection must stay end-to-end trainable.","Because the scorer is conditioned on the text instruction, the method implicitly performs question-aware region retrieval; this could be evaluated as a stand-alone localization task.","If the stop token is reliable, one could expose the selected trajectory to a pathologist as an interactive “evidence path” without extra supervision."],"forward_implications":["Frozen encoder–LLM stacks can be adapted to gigapixel inputs with a single-GPU, single-stage plugin rather than full fine-tuning.","Adaptive, sample-wise token budgets replace fixed retention rates, automatically spending fewer tokens on simple slides and more on complex ones.","Clinical reasoning can improve over the dense baseline once redundant stroma is stripped away.","Quadratic LLM prefill cost scales with intrinsic specimen complexity rather than raw patch count N."],"fun_headline_variants":["45 tokens suffice for whole-slide diagnosis via sequential select","PathSelect routes WSIs to 45 tokens, matches full-context accuracy","Learned scorer prunes pathology slides ~37× with 74% accuracy","Sequential token selection keeps diagnostic signal at 45 patches","Hard Top-K after training cuts WSI context to average 44.86 tokens"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The continuous noisy training path must teach a scorer whose hard discrete selections at inference still carry the same diagnostic information the language-model loss optimized.","fun_headline_variants_meta":{"raw":{"variants":["45 tokens suffice for whole-slide diagnosis via sequential select","PathSelect routes WSIs to 45 tokens, matches full-context accuracy","Learned scorer prunes pathology slides ~37× with 74% accuracy","Sequential token selection keeps diagnostic signal at 45 patches","Hard Top-K after training cuts WSI context to average 44.86 tokens"]},"model":"grok-4.5","effort":"low","cost_usd":0.004429,"raw_usage":{"total_tokens":1362,"prompt_tokens":827,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":44288000,"prompt_tokens_details":{"text_tokens":827,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":457,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":827,"tokens_out":78,"duration_ms":9368,"temperature":1.0,"reasoning_tokens":457,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T17:12:28.420837+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Measure the train–inference selection mismatch directly: if the set of patches retained by Hard Top-K at test time diverges sharply from the soft alive-weighted scores that received gradient, or if accuracy collapses when the diagonal denoiser mask is replaced by full attention, the claimed transfer fails.","supporting_citations":[],"review_version":1}