{"id":"b8eff00b-8119-4fc2-b114-8c7d99c9eccd","arxiv_id":"2608.05808","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"STAIL anchors evolving visual features to frozen LLM text embeddings and rehearses a small image set plus many text descriptions, cutting storage and forgetting in medical class-incremental learning.","lead":"This paper proposes STAIL, a method that helps medical image models keep learning new diagnostic tasks without forgetting old ones by storing short text descriptions of past cases instead of many raw images. The method uses a frozen large language model as a stable semantic anchor and reports consistent improvements across fundus, ultrasound, and X-ray datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's claimed 'consistent and significant' improvements are not supported by the reported error bars: most baseline–dataset–metric comparisons appear within statistical noise, and a few are negative, so the headline claim of consistent significant gains is the weakest point.","rationale":"I agree with the reader that text quality is a real limitation, but the more load-bearing issue is that the paper's own numbers do not establish 'consistent and significant' improvement. The reader flagged overlapping error bars as weakness (a), but then made text availability the weakest assumption; I would rank the significance failure first because it directly falsifies the abstract's central claim even if text were perfect. Additionally, Table 2 shows an internal tension: on US-DATA, the full LSAM is consistently worse than the DSA-only variant (e.g., Replay: 78.74±0.48 vs 85.09±0.65; iCaRL: 80.23±1.94 vs 83.49±0.20), which further undermines the claim that the three losses are complementary and that the full STAIL architecture is responsible for the gains. These two issues together justify the existing CONDITIONAL verdict: the authors need to supply per-seed significance analyses, per-dataset hyperparameter tables, and an explanation of why adding EPA+CSE degrades US-DATA performance before the claims can be accepted.","tokens_in":25253,"tokens_out":15527,"duration_ms":149396,"concrete_test":"Using the three per-seed values behind Table 1 (or a Welch t-test on the reported mean±SD with n=3), compute two-sided p-values for all 36 baseline–dataset–metric comparisons, applying a Benjamini-Hochberg correction. Count how many reach p<0.05 and how many are negative. If fewer than two-thirds are significant, the abstract's 'consistently and significantly improves' must be weakened to 'improves on average in selected benchmarks,' supporting a CONDITIONAL verdict.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; §4.2) is that STAIL 'comprehensively enhances' four rehearsal baselines 'consistently and significantly' across three datasets. The evidence for this, Table 1 with n=3 per cell, shows that the majority of the 36 comparisons (4 baselines × 3 datasets × 3 metrics) have overlapping error bars at roughly the 95% level. For example, on MS-CXR, all three metrics for Replay+STAIL and WA+STAIL are within one standard error of zero effect, and on ODIR, TagFex+STAIL gains are between 0.01 and 0.11 percentage points. Two comparisons move in the wrong direction: BWT-AUC for iCaRL+STAIL on US-DATA (−8.29 vs −8.07) and for TagFex+STAIL on MS-CXR (−2.36 vs −1.42). The only clearly significant effects are concentrated on US-DATA and a few BWT-AUC rows. Because the headline qualifiers 'consistently and significantly' are load-bearing, the absence of per-seed significance testing and multiple-comparison control is the single most direct threat to the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes STAIL, a plug-and-play module for class-incremental learning (CIL) in medical imaging. STAIL replaces pixel-level rehearsal with an asymmetric Semantic Consolidation Buffer (SCB) that stores a few image exemplars plus many paired and randomly sampled text descriptions, and it adds an LLM-derived Semantic Anchoring Mechanism (LSAM) with three losses: developmental semantic anchoring (DSA, Eq. 4), evolutionary plasticity adaptation (EPA, Eq. 5), and consolidated stability enforcement (CSE, Eq. 6). Experiments on ODIR-5K, US-DATA, and MS-CXR claim consistent and significant improvements over four rehearsal baselines (Replay, iCaRL, WA, TagFex) across Avg-AUC, AAA-AUC, and BWT-AUC. The paper also reports ablations, an equal-memory comparison, different LLM choices, text selection strategies, hyperparameter sensitivity, and category-order experiments, plus a theoretical analysis in Appendix B and pseudocode in Appendix C.","tokens_in":25483,"tokens_out":5739,"duration_ms":58576,"significance":"The core idea is interesting and timely: using frozen LLM text embeddings as stationary anchors, and compressing historical memory through text, is a plausible route to privacy-friendlier and more memory-efficient medical CIL. The experimental scope is broad (three heterogeneous modalities, multiple baselines, several ablations), and the equal-memory comparison in Fig. 2 is a genuinely useful check: even when the baseline is given slightly more memory and more visual exemplars, STAIL variants show large gains on US-DATA and MS-CXR. The method is plug-and-play, and code availability is promised. The significance is tempered, however, by the fragility of the statistical evidence, the inconsistency of the full LSAM relative to its own subcomponents, and the overclaimed theoretical contribution. If the statistical and memory-fairness issues are resolved, the work could be a solid contribution to medical continual learning.","major_comments":[{"comment":"The headline claim that STAIL 'consistently and significantly' enhances baselines is not supported by the reported error bars. With only 3 seeds, most of the 36 baseline–dataset–metric comparisons have overlapping means and standard deviations. For example, on MS-CXR, Replay+STAIL gains are only 0.37 (Avg-AUC), 0.01 (AAA-AUC), and 2.85 (BWT-AUC) with standard deviations of 2.71, 1.30, and 4.28, respectively; TagFex+STAIL's BWT-AUC moves in the wrong direction (-2.36 vs -1.42), as does iCaRL+STAIL's BWT-AUC on US-DATA (-8.29 vs -8.07). No per-seed significance testing, confidence intervals, or multiple-comparison correction is provided. The authors should either add paired significance tests (e.g., paired t-test or Wilcoxon across seeds) and report effect sizes with confidence intervals, or qualify the claim to identify the specific settings where gains are statistically reliable.","section":"§4.2, Table 1"},{"comment":"The full LSAM is not consistently better than its own subcomponents, which undercuts the claim that the three losses are complementary. On US-DATA with Replay, DSA-only achieves Avg-AUC 85.09 and AAA-AUC 83.27, while the full DSA+EPA+CSE drops to 78.74 and 81.06; for iCaRL, DSA-only Avg-AUC is 83.49 versus 80.23 for the full model. The paper acknowledges this in the ablation discussion, but it is a load-bearing inconsistency because the final reported model includes all three losses. Please explain why the full model is the deployed configuration, or provide a selection rule, or report the best-performing variant per baseline/dataset.","section":"§4.2, Table 2, ablation"},{"comment":"The main comparisons in Table 1 are not byte-fair: STAIL stores per-class image exemplars plus text exemplars, while the baselines store only images, so the 'fixed per-class memory budget' statement is ambiguous unless text storage is explicitly counted. The equal-memory experiment in Fig. 2 covers only US-DATA and MS-CXR, and for ODIR-5K no byte-level accounting is given (the average text length for ODIR is not reported). Please provide a byte-level memory table for all three datasets and show the main Table 1 results under equal total bytes, or clearly state how the per-class budget was defined and whether the reported improvements change under byte-fair comparison.","section":"§4.1.3, Appendix A, §4.2 equal-memory experiment"},{"comment":"The theoretical claims are overclaimed relative to the content. Proposition 1 is a direct consequence of the triangle inequality under an assumed alignment bound epsilon; it does not prove that the SCB 'reconstructs the data manifold of historical tasks,' as stated in the contribution list. Proposition 2 is an informal statement with no formal statement or proof. Because the paper explicitly highlights this theoretical proof as a contribution, the authors should either provide a genuine formalization (e.g., a coverage or reconstruction guarantee under concrete assumptions on the text distribution) or substantially soften the theoretical claims.","section":"Appendix B, contributions"},{"comment":"The method depends on paired, diagnosis-relevant text for every training image and on the quality of freely sampled extra texts; however, the paper does not analyze robustness to text noise or absence. ODIR-5K was filtered to retain only samples with text, which changes the benchmark, and the random extra texts in Eq. (7) are not quality-filtered. The theoretical guarantee in Appendix B assumes a bounded alignment error epsilon, which may not hold with noisy or semantically misaligned reports. Please add an ablation with corrupted or missing text (e.g., replacing reports with class-level generic descriptions) and report the size and difficulty of the filtered ODIR subset relative to the original.","section":"§4.1.1, Algorithm 1 lines 27-30, Appendix B"}],"minor_comments":[{"comment":"No standard deviations or seed counts are reported in the category-order experiment; please state whether these are single-run results and add error bars or at least note the number of seeds.","section":"Table 4"},{"comment":"The caption says results are averaged over three runs, but the figure does not specify whether error bars/shaded regions are standard deviation or confidence intervals; please clarify.","section":"Fig. 2"},{"comment":"The notation for the CSE loss reuses B_I and P(i) without stating that the text batch is drawn from the SCB; make the data source explicit in the equation or its surrounding text.","section":"Eq. (6)"},{"comment":"Line 29 samples extra texts of size '≤ N_text - N_img', but the buffer update in Eq. (8) adds paired texts plus extra texts; it is unclear whether the final per-class text count is exactly N_text or can be smaller. Clarify the budget semantics.","section":"Algorithm 1, line 29, Eq. (8)"},{"comment":"The 'rare phenomenon' of positive backward transfer (BWT-AUC 2.03 for iCaRL+STAIL on MS-CXR) is reported with a standard deviation of 2.82, so it is statistically indistinguishable from zero; please temper the claim.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is generally well written and the equal-memory experiment is a strong point in its favor, but the statistical support for the central 'consistent and significant' claim is weak, and the memory accounting for ODIR is missing. The ablation inconsistency (full LSAM worse than DSA-only on US-DATA) should be resolved before publication. The theoretical appendix is largely vacuous and should be reframed. These are fixable with additional analysis rather than fundamentally flawed; hence major_revision rather than reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about STAIL is that the core idea is genuinely fresh: instead of storing many images for rehearsal, keep a few image anchors and a lot of text reports, and pin the visual features to frozen LLM embeddings. That is a real departure from the usual pixel-rehearsal or class-level prompt methods, and the equal-memory comparison in Fig. 2 is a thoughtful piece of evidence—they deliberately give the baseline slightly more images and still show STAIL ahead. On the strengths side, the experiments cover three modalities, four rehearsal baselines, LLM choice, text selection strategies, and even a sequence-order robustness test. Code is available. That is a solid empirical package.\n\nThe soft spots are real and load-bearing. The abstract says STAIL 'comprehensively enhances' the baselines 'consistently and significantly.' The stress-test note is right: Table 1 does not support that. With n=3 per cell, most of the 36 comparisons have overlapping error bars. The genuinely significant effects cluster on US-DATA (notably Replay) and a couple of ODIR rows. On MS-CXR, only iCaRL Avg-AUC looks robust. Two BWT comparisons go the wrong way, even if not significantly. So the headline claim overstates the evidence.\n\nAlso, Table 4 reports sequence-order results without any error bars, so the robustness claim there is hard to check. The per-dataset hyperparameter tuning is acknowledged, which is honest but weakens the plug-and-play framing—you need to retune per dataset. The ablation is a bit mixed: on US-DATA, Replay+DSA alone beats the full LSAM by a wide margin on Avg-AUC (85.09 vs 78.74). The paper mentions this, but it undercuts the complementarity story. Finally, the reliance on text quality is a genuine vulnerability: ODIR is filtered to samples with text, and the random extra texts have no quality filter. If the reports are noisy or misaligned with the lesion, the anchoring losses could pull features the wrong way. The appendix's bounded-alignment assumption is doing a lot of work.\n\nNone of this kills the paper. The idea is worth building on, and the negative results are not disqualifying. But the claims need to be toned down, and the authors should provide significance testing or confidence intervals, error bars for Table 4, per-dataset hyperparameter tables, and some analysis of text quality versus anchoring loss behavior.\n\nVerdict: I would send this to peer review—a good referee can help sharpen it—but it needs a major revision before acceptance. I'd bring it to reading group for a critical session, but I wouldn't cite it in my own work yet.","headline":"A genuinely interesting idea—frozen LLM text embeddings as a compressed rehearsal buffer—but the paper's headline claim of consistent, significant gains is not supported by its own error bars.","tokens_in":26115,"tokens_out":5292,"would_cite":false,"duration_ms":44171,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"STAIL replaces most raw-image replay buffers with clinical text anchors and reports consistent gains against forgetting across three medical imaging modalities.","keywords":["class-incremental learning","catastrophic forgetting","medical imaging","semantic consolidation buffer","LLM semantic anchoring","text-anchored feature alignment","multimodal contrastive learning"],"falsifier":"A decisive test would be to shuffle the text descriptions across images within each class before training STAIL: if the gains over the image-only baseline persist under corrupted alignment, the reported improvement is not caused by semantic anchoring, and if they disappear, the frozen text anchors are doing the work. A complementary check is to give the pixel-rehearsal baseline the same total byte budget but let it spend the text portions on additional real images, then compare across three random class orders.","tokens_in":24976,"feed_emoji":"🩻","tokens_out":9632,"duration_ms":91978,"temperature":0.7,"pith_summary":"The paper tries to establish that a medical imaging model can keep learning new diagnostic tasks without forgetting old ones by rehearsing text instead of mostly rehearsing raw images. Its Semantic Consolidation Buffer keeps only a handful of image exemplars per class and pairs them with many clinical text descriptions, which are tens to thousands of times cheaper in bytes. A frozen large language model turns those descriptions into stationary semantic anchors, and three anchoring losses hold the evolving visual feature space close to that fixed semantic space while still allowing new classes to separate. Plugged into four existing rehearsal-based baselines, the module improves average, anytime, and backward-transfer AUC on fundus, ultrasound, and chest X-ray datasets, with average gains of 2.24% in AAA-AUC and 3.55% in BWT-AUC across benchmarks. The reader should care because this points to a memory-efficient and privacy-relevant alternative to pixel-level rehearsal, if paired text is available.","feed_headline":"Text anchors curb forgetting in medical AI by up to 12 points","feed_subtitle":"Few images plus frozen LLM text embeddings preserve old classes across fundus, ultrasound, and chest X-ray tasks.","key_machinery":"The load-bearing object is the asymmetric Semantic Consolidation Buffer (SCB) combined with the LLM-derived Semantic Anchoring Mechanism (LSAM). The SCB keeps a per-class visual core of 5–20 images selected by Herding and a much larger text set of 25–100 descriptions per class, made of the paired reports of the kept images plus randomly sampled extra reports; because a 224×224 image costs hundreds to thousands of times more bytes than a short report, this reconstructs old-task semantics at a fraction of pixel-rehearsal memory. The LSAM then uses a frozen biomedical LLM as a stable coordinate system: a learned projection maps image features into the LLM text space, and the three losses $L_{\\mathrm{DSA}}$, $L_{\\mathrm{EPA}}$, and $L_{\\mathrm{CSE}}$ enforce alignment, contrastive plasticity, and replay of previous geometric constraints. The theoretical argument is that a bounded residual alignment error between image and text features keeps the distance between two visual representations close to the distance between their semantic anchors, so the frozen text space acts as a limit on feature drift.","core_discovery":"The central claim is that catastrophic forgetting in medical class-incremental learning can be reduced by using frozen LLM text embeddings as a stationary coordinate system for the visual feature space. Instead of storing many raw images, STAIL stores a small mean-matching image core plus a large set of paired and randomly sampled reports; during training, a learned projection maps image features into the frozen text space, and the three LSAM losses — developmental semantic anchoring, evolutionary plasticity adaptation, and consolidated stability enforcement — keep old-class geometry from drifting while refining new-class separability. The reported result is that STAIL works as a plug-and-play module: it improves Replay, iCaRL, WA, and TagFex on ODIR-5K, US-DATA, and MS-CXR, with the largest single improvements of about +8.10 percentage points in Avg-AUC on US-DATA for Replay and +12.00 percentage points in BWT-AUC on ODIR-5K for WA, while using less memory than the image-heavy buffers it replaces. The paper also argues theoretically that bounded alignment error between visual and semantic features bounds the distortion of inter-class distances, which is the mechanism behind the stability it observes.","pith_inferences":["The same text-anchored replay recipe should transfer to nonmedical continual learning wherever paired image–text data already exist, such as satellite imagery with captions or inspection logs; running STAIL on a standard class-incremental benchmark with a general-purpose LLM under byte-equal memory would test this directly.","The privacy advantage is only partial: trading raw images for free-text radiology reports moves the sensitive data from pixels to patient language, so a deployment would still need de-identification or synthetic text before the privacy claim is complete.","Positive backward transfer on MS-CXR hints that frozen text anchors can act as a deliberate review mechanism: scheduling rare classes early and giving them dense text memory might be a better curriculum than the uniform task orders the paper evaluates."],"forward_implications":["Existing rehearsal-based baselines can adopt STAIL without changing their own loss terms; the module only adds a text stream and three weighted anchor losses, and the reported gains hold across four baselines and three imaging modalities.","Because a standard 224×224 image costs roughly 462–3,369 times more memory than the average text report in the authors' datasets, the same byte budget can support far denser semantic memory when images are partially replaced by text.","The three losses separate the stability and plasticity jobs: $L_{\\mathrm{DSA}}$ and $L_{\\mathrm{CSE}}$ protect old classes while $L_{\\mathrm{EPA}}$ specializes new ones, so practitioners can reweight them for datasets dominated by drift or dominated by new-class confusion.","On the long-tailed MS-CXR benchmark, STAIL can reach positive backward transfer (BWT-AUC +2.03% for iCaRL), meaning later tasks can strengthen, not merely preserve, earlier knowledge."],"supporting_citations":[{"why":"Supplies the rehearsal baseline and the nearest-mean-of-exemplars classifier that STAIL must improve and that defines the exemplar-buffer comparison.","marker":"[31]"},{"why":"Weight-aligning baseline whose classifier-bias correction is one of the methods STAIL is plugged into and improves.","marker":"[54]"},{"why":"Task-agnostic guided feature expansion baseline; the strongest architecture STAIL must be evaluated against.","marker":"[55]"},{"why":"Herding dynamical weights method used for selecting the few image exemplars in the SCB visual core.","marker":"[47]"},{"why":"Frozen biomedical LLM text encoder that produces the stationary semantic anchors used by all three LSAM losses.","marker":"[18]"},{"why":"MS-CXR chest X-ray image–text dataset used to test long-tailed and text-quality behaviour.","marker":"[5]"},{"why":"US-DATA ultrasound image–report dataset used to test multi-organ, fine-grained incremental learning.","marker":"[22]"},{"why":"ODIR-5K fundus dataset used to evaluate ophthalmic class-incremental learning, filtered to samples with text.","marker":"[37]"},{"why":"Defines the AAA-AUC metric that measures sustained performance across the whole incremental stream.","marker":"[7]"},{"why":"Defines the BWT-AUC metric that measures how much old-task accuracy is lost after later tasks.","marker":"[8]"}],"fun_headline_variants":["Text anchors cut medical AI forgetting by up to 12 points","LLM text anchors stabilize learning, curb forgetting in medical AI","Frozen LLM text anchors reduce forgetting in medical imaging","Semantic text anchoring cuts memory and forgetting in medical AI","Text anchors slash forgetting in medical AI, saving storage"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains rely on every training image having a paired text description that actually describes the clinically relevant content and that the frozen LLM embeds faithfully; if reports are missing, noisy, or semantically misaligned with the image, the anchoring losses can pull visual features toward the wrong points and the theoretical bound on alignment error no longer holds.","fun_headline_variants_meta":{"raw":{"variants":["Text anchors cut medical AI forgetting by up to 12 points","LLM text anchors stabilize learning, curb forgetting in medical AI","Frozen LLM text anchors reduce forgetting in medical imaging","Semantic text anchoring cuts memory and forgetting in medical AI","Text anchors slash forgetting in medical AI, saving storage"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000932,"raw_usage":{"total_tokens":4038,"prompt_tokens":1045,"completion_tokens":2993,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":2911}},"tokens_in":661,"tokens_out":2993,"duration_ms":21508,"temperature":1.0,"reasoning_tokens":2911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T23:05:09.005581+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be to shuffle the text descriptions across images within each class before training STAIL: if the gains over the image-only baseline persist under corrupted alignment, the reported improvement is not caused by semantic anchoring, and if they disappear, the frozen text anchors are doing the work. A complementary check is to give the pixel-rehearsal baseline the same total byte budget but let it spend the text portions on additional real images, then compare across three random class orders.","supporting_citations":[{"cited_title":"iCaRL: Incremental Classifier and Representation Learning, in: Proceedings oftheIEEEConferenceonComputerVisionandPatternRecognition, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the rehearsal baseline and the nearest-mean-of-exemplars classifier that STAIL must improve and that defines the exemplar-buffer comparison."},{"cited_title":"Maintaining discriminationandfairnessinclassincrementallearning,in:Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Weight-aligning baseline whose classifier-bias correction is one of the methods STAIL is plugged into and improves."},{"cited_title":"Task-Agnostic Guided Feature Expansion for Class-Incremental Learning, in: Pro- ceedings of the Computer Vision and Pattern Recognition Confer- ence, pp","cited_arxiv_id":null,"evidence_quote":"Task-agnostic guided feature expansion baseline; the strongest architecture STAIL must be evaluated against."},{"cited_title":"Herding dynamical weights to learn, in: Pro- ceedings of the 26th Annual International Conference on Machine Learning, pp","cited_arxiv_id":null,"evidence_quote":"Herding dynamical weights method used for selecting the few image exemplars in the SCB visual core."},{"cited_title":"Ultrasound report generation with cross-modality feature alignmentviaunsupervisedguidance","cited_arxiv_id":null,"evidence_quote":"US-DATA ultrasound image–report dataset used to test multi-organ, fine-grained incremental learning."},{"cited_title":"In- ternational Competition on Ocular Disease Intelligent Recognition","cited_arxiv_id":null,"evidence_quote":"ODIR-5K fundus dataset used to evaluate ophthalmic class-incremental learning, filtered to samples with text."},{"cited_title":"Toward Bidirectional Adaptability for Few-Shot Class-Incremental LearningWithForward-BackwardKnowledgeTransfer","cited_arxiv_id":null,"evidence_quote":"Defines the BWT-AUC metric that measures how much old-task accuracy is lost after later tasks."}],"review_version":1}