{"id":"0b1c8171-37b9-46a3-b085-b2043bcda5c6","arxiv_id":"2505.20933","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"InfoComp learns per-task private prompts and one shared prompt with parameter-alignment and SimSiam-style losses, reporting consistent accuracy gains over prior state-of-the-art continual text classification methods on 5- and 15-task benchmarks.","lead":"This paper presents InfoComp, a prompt-tuning method for continual text classification that learns a private prompt for each task plus one shared prompt across tasks, trained with two similarity-based losses inspired by mutual information. The authors report consistent accuracy gains over prior methods on five-task and fifteen-task benchmarks without data replay, with average improvements between 0.9 and 2.7 points.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 4's reported average improvements do not match its per-cell improvements, and the reimplemented ProgPrompt baseline differs from the cited numbers by up to 5.2 points, so the headline 3.7-point gain is not robustly anchored.","rationale":"The reader's weakest assumption was baseline comparability, and I agree that is the core load-bearing premise. I found a sharper, checkable defect in the same premise: Table 4's Improve row does not aggregate consistently. The average of the per-cell improvements is 2.2/2.3/1.9, not the reported 3.7/2.7/1.9. The 20-shot 3.7 is produced by subtracting the average of the authors' lower reimplemented ProgPrompt from InfoComp's average, while the per-cell values use the better of cited or reimplemented ProgPrompt. In addition, the reimplemented ProgPrompt and the cited ProgPrompt differ by as much as 5.2 points in Order7/20, which is larger than several of InfoComp's claimed margins. This means the headline gain can be moved by more than a point simply by choosing which ProgPrompt column to compare against. The issue is fixable through code release and a consistent recomputation, so it does not justify rejection; it supports the conditional verdict and demands verification before the empirical claim is accepted.","tokens_in":16836,"tokens_out":9343,"duration_ms":95400,"concrete_test":"Recompute Table 4 with a single, pre-specified baseline rule: compare every InfoComp entry to the authors' own ProgPrompt re-implementation using identical prompt length (35+5 tokens), optimizer, validation splits, and the same three seeds, and verify that the Avg column equals the mean of the per-order improvements. If the 20-shot average gain falls from 3.7 to about 2.2, or loses the paired t-test at p<0.05, the headline advantage is an artifact of mixing cited and reimplemented baselines.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on Table 4's 'Improve' row, and that row is internally inconsistent. The per-cell improvements (Order5: 1.1/1.7/1.5; Order6: 3.5/2.3/2.4; Order7: 2.6/3.0/1.8) average to 2.2/2.3/1.9, but the reported Avg improvements are 3.7/2.7/1.9. The 20-shot '3.7' equals InfoComp's average (57.9) minus the reimplemented ProgPrompt* average (54.2), whereas the per-cell values compare InfoComp with the stronger of cited or reimplemented ProgPrompt for each cell; the 200-shot '2.7' uses cited ProgPrompt. Meanwhile, the reimplemented ProgPrompt deviates from the cited numbers by up to 5.2 points (Order7/20: 51.9 vs 57.1), a gap larger than many claimed gains. Because no per-run values, standard deviations, or code are provided, the headline 'up to 3.7 points' cannot be distinguished from the choice of comparison baseline. If the comparison protocol is fixed to one consistent baseline, the aggregate 20-shot gain drops to roughly 2.2 points, and individual improvements may fall within run-to-run noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces InfoComp, a rehearsal-free continual text classification (CTC) method that maintains per-task private prompts (P-Prompt) and a shared prompt (S-Prompt), trained with two auxiliary objectives presented as mutual-information maximization (Eqs. (5) and (7)) and combined in Eq. (8). Experiments on a 5-task standard benchmark and 15-task long-sequence benchmark (Tables 3 and 4) report accuracy gains over ProgPrompt and other baselines, with ablations in Table 5 supporting the contribution of each component. The central empirical claim is that InfoComp outperforms prior state-of-the-art methods without data replay, with average gains of 0.9 points on the 5-task benchmark and up to 3.7 points on the 15-task few-shot setting.","tokens_in":17171,"tokens_out":7814,"duration_ms":79123,"significance":"If the results hold under a controlled comparison, InfoComp would be a simple, parameter-efficient, replay-free alternative to progressive prompt concatenation, and the constant prompt length is a practical advantage over ProgPrompt. The ablation study is a genuine strength: it isolates the P-Prompt, S-Prompt, and each auxiliary loss. However, the information-theoretic framing currently rests on heuristic similarity losses rather than demonstrable mutual-information objectives, and the headline improvement is sensitive to the choice of comparison baseline. The empirical claims are promising but need verification before the paper can be accepted.","major_comments":[{"comment":"The 'Improve' row is computed with an inconsistent baseline. The per-cell improvements (Order5: 1.1/1.7/1.5; Order6: 3.5/2.3/2.4; Order7: 2.6/3.0/1.8) average to about 2.2, while the reported averages 3.7, 2.7, and 1.9 are obtained by comparing InfoComp's average to the reimplemented ProgPrompt* for 20 shots (57.9 - 54.2) and to the cited ProgPrompt for 200 and 1000 shots (69.6 - 66.9 and 71.2 - 69.3). Some per-cell entries also use the stronger of the cited and reimplemented ProgPrompt values. Please fix the table by choosing one comparison protocol and reporting the average of the per-cell differences; the claim of 'up to 3.7 points' is not supported by the current numbers, although a consistent comparison still leaves a positive average gain of roughly 2.2 points.","section":"Table 4"},{"comment":"The paper asserts that maximizing the inner product in Eq. (5) maximizes I(θPk; θhead|X), and that the SimSiam-style objective in Eq. (7) maximizes I(V_i^k; V_i^{k'}), but no derivation or variational bound is provided. A bilinear similarity term does not generally equal or bound mutual information, and the SimSiam objective is a positive-only alignment loss rather than an information-theoretic estimator. Since the method's name and its two main contributions are explicitly information-theoretic, either provide a formal derivation (e.g., an InfoNCE-style lower bound) or reframe these terms as similarity-based regularizers. As written, the central conceptual claim is unsupported.","section":"Section 4.2, Eqs. (5)-(7)"},{"comment":"The † significance markers are unverifiable as reported. The manuscript states only that a paired t-test at the 0.05 level was used; it does not specify the pairing unit (task order? seed?), the number of runs used in the test, the degrees of freedom, or the observed standard deviations. Since all results are averages of 3 runs and no per-run values are given, the statistical significance claims cannot be checked. Please report per-run results with standard deviations and the exact test protocol.","section":"Section 5.3, Tables 3 and 4"},{"comment":"Most baseline results are cited from prior papers rather than re-run under the same protocol, and the reimplemented ProgPrompt* deviates from the cited numbers by up to 5.2 points (Order7/20: 57.1 vs 51.9), a gap larger than many of the claimed improvements. This makes it hard to rule out configuration differences (prompt length, epochs, validation splits, seeds) as an alternative explanation for the observed gains. Please either re-run all baselines under the identical protocol used for InfoComp, or provide explicit evidence that the cited numbers are compatible with that protocol.","section":"Section 5.2, Table 4"}],"minor_comments":[{"comment":"The implementation details mention a 'G-Prompt length' of 5, but no G-Prompt is defined anywhere in the paper; this appears to be a typo for the S-Prompt length.","section":"Section 5.4"},{"comment":"The stop-gradient operation, which is central to SimSiam and is shown in Figure 3, does not appear in the equations; please state explicitly whether gradients are blocked on V_i^{k'} in Eq. (7).","section":"Eqs. (6)-(7) and Figure 3"},{"comment":"The expression θhead^T W1 θP^T requires the prompt parameters and classifier weights to be arranged as vectors; please specify how these parameters are flattened and give the shape of W1.","section":"Eq. (5)"},{"comment":"The sentence reporting 'an improvement of 1.6, 2.3, 2.0, and 2.3 over ProgPrompt' is internally ambiguous: the second value 2.3 matches the reimplemented ProgPrompt* (80.1 - 77.8), while the cited ProgPrompt row gives 80.1 - 77.7 = 2.4. Please clarify which baseline is being compared.","section":"Section 6.1.1"},{"comment":"The paper would be easier to reproduce if the authors released code, seeds, and per-run results for the reimplemented baselines, especially because the comparison is sensitive to baseline configuration.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a journal on neural networks and continual learning, but the current empirical protocol needs substantial strengthening before publication. The Table 4 inconsistency is the most urgent issue, followed by the unsupported mutual-information claims. The method itself is simple and plausible, so I would not reject it, but the claims as written go beyond the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper before citing it: the method is real and the core ablation story holds up, but the long-sequence comparison table has an internal inconsistency that undercuts the headline gain. If that inconsistency is fixed, the method still looks competitive, just less dramatic.\n\nWhat's new: InfoComp combines two prompt types—per-task private prompts and a shared prompt—plus two auxiliary losses. The private-prompt loss is an inner product between prompt and classifier parameters, which echoes InfoPrompt; the shared-prompt loss is SimSiam-style consistency between current and previous S-Prompt representations. Neither loss is novel, but the specific combination for rehearsal-free continual text classification is new, and the paper tests it on standard and long-sequence benchmarks. The ablations are clean: removing either prompt type or either loss drops performance by roughly 1 to 8 points, and the gains are consistent across seven task orders. On the standard benchmark the numbers are internally consistent, and the average gains over SLM (0.4 to 1.3 points) are modest and believable.\n\nThe soft spots are real but fixable. First, the information-theoretic framing is mostly dressing. Equations (5) through (7) say that maximizing inner products or SimSiam-style similarity maximizes mutual information, but no derivation or bound is given. That is a presentation flaw rather than a fatal one; the losses are still reasonable heuristics. Second, Table 4's improvement row is inconsistent. The per-cell improvements for Orders 5-7 average to about 2.2, 2.3, and 1.9 points for the three sample sizes, but the reported averages are 3.7, 2.7, and 1.9. The 3.7 figure compares InfoComp to the reimplemented ProgPrompt only, while per-cell numbers seem to use the stronger of cited or reimplemented baselines. Also, the reimplemented ProgPrompt deviates from the cited numbers by up to 5.2 points, so the comparison protocol is not stable. There are no standard deviations, no per-run values, and no code, so the paired t-test significance claims cannot be checked. These are exactly the things that peer review should demand.\n\nWho should read this? Anyone working on rehearsal-free continual text classification or prompt-based continual learning. The method is simple to reproduce and the empirical evidence, while not overwhelming, is genuine. I would send it to peer review, but with a request for major revision that fixes the baseline consistency and variance reporting. If those come back clean, it would be a decent contribution.","headline":"A real rehearsal-free prompt-based CTC method with clean ablations, but the headline 3.7-point gain rests on an internally inconsistent comparison row and unchecked baseline numbers.","tokens_in":17674,"tokens_out":1607,"would_cite":true,"duration_ms":16860,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InfoComp learns two complementary prompts to classify a stream of text tasks without replaying old data.","keywords":["continual text classification","catastrophic forgetting","forward knowledge transfer","prompt tuning","mutual information","complementary learning systems","rehearsal-free continual learning","parameter-efficient fine-tuning"],"falsifier":"Re-run the cited baselines and InfoComp on the same four task orders and three 15-task orders with identical prompt lengths, epochs, seeds, validation splits, and test sets, and compare paired differences with the reported $p<0.05$ test; if the average gains over the strongest baselines fall below the significance threshold under those controlled conditions, the claim of consistent state-of-the-art improvement is falsified.","tokens_in":16661,"feed_emoji":"🧠","tokens_out":7379,"duration_ms":69575,"temperature":0.7,"pith_summary":"InfoComp claims that continual text classification is best served by splitting the prompt into two roles: a private prompt per task that absorbs task-specific knowledge, and a shared prompt that accumulates task-invariant knowledge across all tasks. The paper argues that previous prompt-based continual learners either ignore shared knowledge or pay a growing computational cost by concatenating all past prompts. InfoComp adds two information-theoretic objectives that push the private prompt to align with its task classifier and push the shared prompt to keep representations consistent with its earlier state. The reported result is that this design, without any data replay, consistently beats prior methods on the standard five-task benchmark (average accuracy 80.0, up 0.9 over the best baseline) and on the fifteen-task long-sequence benchmark (up to 3.7 points in the 20-sample few-shot setting). A sympathetic reader would care because it suggests a cheap, privacy-friendly route to lifelong text classification that keeps prompt length fixed rather than growing with the number of tasks.","feed_headline":"Two-prompt method tops continual text classification without replay","feed_subtitle":"InfoComp pairs a private per-task prompt with a shared prompt, gaining up to 3.7 points over prior SOTA","key_machinery":"The load-bearing object is the complementary prompt pair: a private P-Prompt per task and a single shared S-Prompt, both soft prompts fed into the frozen pretrained language model. The information-theoretic machinery that carries the argument is the pair of surrogate mutual-information objectives: the inner-product loss $L_{p\\text{-info}}^k = -\\theta_{\\text{head}}^{k\\,T} W_1 \\theta_{P_k}^T$ is used as a gradient-friendly proxy for $I(\\theta_{P_k}; \\theta_{\\text{head}}^k \\mid X_i^k)$, and a SimSiam-style consistency loss $L_{s\\text{-info}}^k = -(W_q V_i^k)^T V_i^{k\\prime}$ is used as a positive-only contrastive proxy for $I(V_i^k; V_i^{k\\prime})$, with a stop-gradient on the previous S-Prompt representation. These two losses let the same fixed-length prompt architecture push task-specific information into the private prompt and shared information into the shared prompt, which is what prevents forgetting and enables forward transfer.","core_discovery":"The central claim is that explicitly learning two complementary prompt spaces improves continual text classification. For each task $T_k$ the model trains a task-specific P-Prompt $P_k$ and a shared S-Prompt $S$, with cross-entropy loss $L_k$ on the input $[P_k, S, X_i^k]$ while the PLM stays frozen. Two mutual-information losses are added: $L_{p\\text{-info}}^k$ maximizes the inner product between the P-Prompt parameters and the task classifier parameters, which the paper reads as increasing $I(\\theta_{P_k}; \\theta^k_{\\text{head}} \\mid X_i^k)$ and thereby strengthening task-specific knowledge; $L_{s\\text{-info}}^k$ maximizes agreement between representations of the same input under the current and the frozen previous S-Prompt, following the SimSiam objective, which preserves task-invariant knowledge across the sequence. The total objective is $L_{\\text{overall}}^k = L_k + \\lambda_1 L_{p\\text{-info}}^k + \\lambda_2 L_{s\\text{-info}}^k$. The paper reports consistent gains over the strongest prompt-based baselines on four orderings of the standard five-task CTC benchmark and three orderings of a 15-task benchmark, with differences judged significant at $p < 0.05$.","pith_inferences":["Editorial inference: the S-Prompt consistency loss is a form of self-distillation through prompt states, so InfoComp may transfer to other continual settings with a frozen encoder, such as continual named entity recognition or continual question answering.","Editorial inference: because the method is model-agnostic and only trains prompt parameters, a testable extension is applying it to decoder-only language models or multimodal encoders; the inner-product mutual-information surrogate may need recalibration when the classifier head is replaced by a language-model head.","Editorial inference: the claimed gains rest on comparisons to baselines cited from earlier papers, so a fully controlled re-run with identical prompt lengths, epochs, seeds, and validation splits is the natural next check.","Editorial inference: a direct probe of whether the S-Prompt truly stores task-invariant knowledge would be to freeze it after training and evaluate zero-shot or few-shot transfer to an unseen task; the paper does not report such a probe."],"forward_implications":["InfoComp keeps total prompt length constant as the number of tasks grows, because only the current P-Prompt and the shared S-Prompt are used; this removes the linear prompt-list growth and its associated computational cost.","Rehearsal-free continual text classification can outperform replay-based methods: InfoComp beats the episodic-memory baselines on the standard benchmark while storing no previous examples.","The ablation results imply that both prompt types and both mutual-information losses contribute; removing either prompt or either loss lowers accuracy on the 15-task orders.","The gains extend to few-shot regimes: at 20 samples per class the method reports a 3.7-point average improvement over the strongest baseline across three 15-task orders."],"supporting_citations":[{"why":"Defines the progressive-prompt CTC setup and supplies the strongest baseline InfoComp is compared against.","marker":"[13]"},{"why":"Supplies the episodic memory baseline MBPA++ and the data splits for the standard CTC benchmark.","marker":"[2]"},{"why":"Supplies the IDBR baseline and the training/validation protocol used for the CTC benchmarks.","marker":"[3]"},{"why":"Motivates the use of mutual information in soft-prompt tuning and the claim that prompts alone under-encode task information.","marker":"[20]"},{"why":"Supplies the SimSiam positive-only contrastive objective adapted for the S-Prompt consistency loss.","marker":"[67]"},{"why":"Provides the complementary learning systems theory behind the P-Prompt/S-Prompt split.","marker":"[15, 16]"}],"fun_headline_variants":["InfoComp: private plus shared prompts beat continual text classification","InfoComp boosts continual text classification without replay","Dual prompts cut forgetting: InfoComp improves continual text classification","InfoComp tops continual text classification with complementary prompts","No replay: InfoComp's twin prompts advance continual text classification"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the baseline numbers taken from earlier papers are directly comparable to InfoComp, since most baselines in Tables 3 and 4 were not re-run under the same prompt lengths, epochs, seeds, and validation protocols; if those configurations differ, the reported gains could be configuration effects rather than a property of the method.","fun_headline_variants_meta":{"raw":{"variants":["InfoComp: private plus shared prompts beat continual text classification","InfoComp boosts continual text classification without replay","Dual prompts cut forgetting: InfoComp improves continual text classification","InfoComp tops continual text classification with complementary prompts","No replay: InfoComp's twin prompts advance continual text classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00108,"raw_usage":{"total_tokens":4576,"prompt_tokens":1061,"completion_tokens":3515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":677,"completion_tokens_details":{"reasoning_tokens":3437}},"tokens_in":677,"tokens_out":3515,"duration_ms":27652,"temperature":1.0,"reasoning_tokens":3437,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:43:34.126915+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the cited baselines and InfoComp on the same four task orders and three 15-task orders with identical prompt lengths, epochs, seeds, validation splits, and test sets, and compare paired differences with the reported $p<0.05$ test; if the average gains over the strongest baselines fall below the significance threshold under those controlled conditions, the claim of consistent state-of-the-art improvement is falsified.","supporting_citations":[{"cited_title":"Razdaibiedina, Y","cited_arxiv_id":null,"evidence_quote":"Defines the progressive-prompt CTC setup and supplies the strongest baseline InfoComp is compared against."},{"cited_title":"de Masson D’Autume, S","cited_arxiv_id":null,"evidence_quote":"Supplies the episodic memory baseline MBPA++ and the data splits for the standard CTC benchmark."},{"cited_title":"Huang, Y","cited_arxiv_id":null,"evidence_quote":"Supplies the IDBR baseline and the training/validation protocol used for the CTC benchmarks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SimSiam positive-only contrastive objective adapted for the S-Prompt consistency loss."}],"review_version":1}