{"id":"5c97a0c5-50e8-4d05-a0b0-c9dccd8c497a","arxiv_id":"2412.08285","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Per-task prompt pools plus Gaussian latent replay let a frozen BERT model learn new relations without storing old data, matching rehearsal-based baselines.","lead":"This paper introduces WAVE-CRE, a rehearsal-free method for continual relation extraction that assigns each task its own prompt pool and replays synthetic latent representations instead of storing past examples. It reports accuracy competitive with rehearsal-based systems on FewRel and TACRED, which matters because it suggests strong continual learning is possible without data storage or privacy costs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The rehearsal-free claim rests on unvalidated per-relation Gaussian fits in Section 3.2; a poor fit would break the task predictor and relation classifier, and no diagnostics or ablations support this core assumption.","rationale":"The reader identified the unvalidated Gaussian assumption in Section 3.2 as the weakest assumption, and I agree. The paper's central claim is that WAVE-CRE achieves near-parity with rehearsal-based methods without storing raw data, but the only mechanism for preserving past relation knowledge is generative replay from per-relation Gaussian distributions. Both the relation classifier and the task predictor are trained on samples from these distributions, so any mismatch between synthetic and real representations directly degrades the two components that enable continual learning. The paper provides no empirical evidence that the Gaussian fits are adequate, no comparison against a nonparametric replay oracle, and no memory analysis for full versus diagonal covariance. Despite these gaps, the method is clearly described and the reported margins over rehearsal-free baselines are consistent, so the appropriate verdict remains CONDITIONAL rather than REJECT. The proposed oracle-replay test would settle whether the Gaussian assumption is actually load-bearing or merely a theoretical concern.","tokens_in":13331,"tokens_out":4168,"duration_ms":46026,"concrete_test":"Run the full WAVE-CRE pipeline on FewRel/TACRED, but replace the Gaussian replay in Eqs. (16)-(17) with an oracle replay that uses the actual stored z and q vectors from all past relations (or a random subset of them), keeping all other components fixed. Compare T10 accuracy and task prediction precision (Table 4) to the Gaussian-replay variant. If oracle replay improves final accuracy by more than 2 points, the Gaussian assumption is the limiting factor and the rehearsal-free claim is not substantiated; if it does not, the Gaussian fit is adequate. As a complementary check, compute the maximum mean discrepancy (MMD) between real and Gaussian-sampled z per relation to quantify fit quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 models each relation's prompted representation z and query q as a single Gaussian (Eqs. 16-17), and these synthetic samples are the only mechanism for retaining past knowledge. The task predictor (Eq. 18) and relation classifier (Eq. 19) are trained exclusively on samples drawn from these Gaussians, while at test time they receive real q and z. If the true latent distributions are multi-modal or skewed, the synthetic replay will not match test-time features, causing the task predictor to choose the wrong prompt pool and the classifier to misclassify old relations. The paper reports no fit diagnostics and no ablation that replaces Gaussian replay with actual stored features or a more flexible generative model, so the central rehearsal-free claim is unsupported at its most critical point. Moreover, if full covariance matrices are stored as written, the memory cost is O(D^2) per relation (~376 MB for 80 relations at D=768), which weakens the stated memory-efficiency motivation; using diagonal covariance would make the fit even more fragile.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes WAVE-CRE, a rehearsal-free continual relation extraction method that combines task-specific prompt pools with per-relation Gaussian generative models. For each task, a dedicated prompt pool is learned with key-query selection (Section 3.1). After training, per-relation Gaussian distributions are fitted to prompted representations and query features (Section 3.2), and synthetic samples from these Gaussians are used to train a task predictor and the shared relation classifier (Section 3.3). Experiments on FewRel and TACRED report consistent gains over prompt-based and rehearsal-free baselines, and near-parity or slight superiority over rehearsal-based methods on the final task (Table 1).","tokens_in":13540,"tokens_out":5907,"duration_ms":66226,"significance":"If the reported results are reliable, WAVE-CRE offers a genuinely rehearsal-free alternative with competitive accuracy, and the within-task variance framing of prompt pools is a useful conceptual contribution. The paper covers two standard benchmarks, consistently improves over rehearsal-free baselines, and includes ablations isolating the prompt-pool design. However, the central empirical claim is not yet fully supported: the Gaussian generative assumption is unvalidated, no statistical significance is reported for the key comparisons, and a core ablation is run only on one dataset in a simplified setting. These gaps are load-bearing because the method's rehearsal-free mechanism depends entirely on the fidelity of the fitted Gaussians.","major_comments":[{"comment":"The Gaussian assumption is load-bearing and unvalidated. The task predictor (Eq. 18) and relation classifier (Eq. 19) are trained exclusively on samples drawn from the per-relation Gaussians, while at test time they receive real q and z. If the true latent distributions are multi-modal or skewed, the synthetic replay will misrepresent past relations, degrading both the task predictor and the relation classifier. The paper reports no fit diagnostics (e.g., goodness-of-fit, visualization, or comparison to stored-feature replay) and no ablation with a more flexible generative model. Additionally, storing full covariance matrices costs O(D^2) per relation (about 376 MB for 80 relations at D=768), which weakens the stated memory-efficiency motivation; switching to diagonal covariance would make the fit even more fragile. Please validate the Gaussian assumption or bound the harm by ablating synthetic replay against stored-feature replay.","section":"Section 3.2, Eqs. (16)-(17)"},{"comment":"The central empirical claims are not supported by statistical evidence. The paper reports \"mean accuracy on 5 different random seeds\" but gives no standard deviations, confidence intervals, or significance tests. The key comparisons are close: on TACRED T10, WAVE-CRE (78.7) is below CRE-DAS (79.1), and on FewRel T10 the margin over CDec+ACA is only 0.2 points (85.0 vs 84.8). Without variance information, the claims of \"superior performance\" and \"nearly equivalent to rehearsal-based baselines\" are not established. Please report error bars and, where appropriate, paired significance tests across seeds.","section":"Table 1, Section 4.1 (Evaluation Metrics)"},{"comment":"The ablation supporting the task-specific prompt pool is run only on TACRED in the task-incremental setting, and the 1.8-point improvement on the final task (85.2 vs 83.4) has no confidence interval. Since the prompt pool is a core component of the proposed method, the ablation should be repeated on both datasets and in the class-incremental setting (the main evaluation protocol), or the claim about within-task variance should be weakened accordingly.","section":"Section 4.3, Table 2"},{"comment":"The hyperparameters M (prompt pool size), K (number of selected prompts), and λ (regularization weight) are not reported; the paper states only that they were tuned via random search. This prevents reproducibility and makes it difficult to assess sensitivity to these choices. Please report the selected values and the search ranges.","section":"Section 3.1, Eq. (15) and Section 4.1 (Implementation Details)"}],"minor_comments":[{"comment":"In the TACRED block, the EA-EMR value at T5 is reported as \"24\" while all other values use two decimal places; please format consistently.","section":"Table 1"},{"comment":"The sentence \"Future works may explore alternative generative models\" is the only acknowledgment of the Gaussian assumption's limitation; please add a sentence explicitly noting the risk of distribution mismatch between synthetic and real representations at test time.","section":"Section 3.2"},{"comment":"The method name appears with varying spacing as \"W A VE-CRE\", \"WAVE-CRE\", and \"W A VE-CRE\"; please unify the notation throughout the text, figures, and captions.","section":"Throughout"},{"comment":"Step 3 uses the notation \"xB ∈ Dt\" but Dt is not listed among the inputs; please clarify that Dt denotes the t-th training dataset.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The method's MoE-prefix connection is motivated almost entirely by Le et al. (2024a), a NeurIPS paper by several of the same authors. This is acceptable as background, but the incremental contribution over that prior work (task-specific prompt pools plus Gaussian replay, rather than the MoE interpretation itself) should be made more explicit in the revision. The manuscript's formatting and copyright line suggest an AAAI-style submission; if it is being considered by a journal, the reproducibility details and related-work discussion will need expansion beyond the current camera-ready format."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a plausible paper and a real empirical step for rehearsal-free continual relation extraction. The combination of per-task prompt pools with Gaussian latent replay is new in the CRE literature, and the experiments are consistent: WAVE-CRE beats the rehearsal-free baselines by a wide margin on both FewRel and TACRED and lands within a couple points of the rehearsal-based methods, even topping them on FewRel T10. The ablations on the prompt pool and on the number of experts per prompt are sensible and support the design choices.\n\nThe main soft spot is exactly where the stress-test lands: Section 3.2 fits one Gaussian per relation for both the prompted representation and the query, and then trains the task predictor and classifier entirely on samples drawn from those Gaussians. At test time the model sees real features. If those distributions are multi-modal or skewed, the synthetic replay misleads both heads, and the rehearsal-free claim rests on an assumption the paper never checks. There are no fit diagnostics, no comparison against storing real features, and no ablation with a more flexible generative model. The memory claim is also shaky: storing full covariance matrices per relation is O(D^2), which for 80 relations at D=768 is hundreds of MB, not obviously better than a small buffer. The paper should either use diagonal covariance with analysis or report the actual memory.\n\nMinor issues: no error bars or significance tests (they report mean over 5 seeds only), hyperparameter values are not listed, and the prompt-pool ablation is only on TACRED task-incremental. The MoE-prefix connection is self-cited and only motivational, so it's not circular; the empirical results stand on their own.\n\nThis paper deserves a serious referee. It is clearly written, the method is reproducible in principle, and the empirical contribution is meaningful for the CRE subfield. The revisions I'd demand are the Gaussian diagnostics/ablation and the memory accounting, plus the missing statistics. I'd be inclined to conditionally accept after that.","headline":"A competently executed rehearsal-free CRE method with a genuinely new prompt-pool plus Gaussian replay combination, whose load-bearing generative assumption needs more evidence before the memory-efficiency claim can be trusted.","tokens_in":14082,"tokens_out":2779,"would_cite":true,"duration_ms":28493,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Per-task prompt pools plus Gaussian replay match memory-based continual relation extraction","keywords":["continual relation extraction","catastrophic forgetting","rehearsal-free continual learning","prompt pool","prefix-tuning","mixture of experts","generative replay","task predictor"],"falsifier":"On a relation whose encoded examples form two or more well-separated clusters, fit a two-component Gaussian mixture to the training representations from $D^r_z$. If the relation classifier trained on samples from the single fitted Gaussian loses more than a few points relative to one trained on the mixture, while the mixture version does not, then the single-Gaussian assumption is the weak link and the replay will misrepresent that relation.","tokens_in":13138,"feed_emoji":"🧠","tokens_out":10174,"duration_ms":94213,"temperature":0.7,"pith_summary":"The paper argues that continual relation extraction can shed the memory buffer without losing accuracy, provided each task keeps a private prompt pool and old relations are replayed as synthetic latent samples rather than stored sentences. Existing prompt-based methods fail, the paper says, because prompt selection is inaccurate, shared parameters still forget, and neither cross-task nor within-task variation is handled well. WAVE-CRE assigns each task its own pool of one-expert prompts, chosen by matching a query vector to learned keys, and fits a separate Gaussian to the encoded representations and to the query vectors of every relation seen so far. Those Gaussians generate the training data for the relation classifier and for a task predictor that selects the correct pool at test time, so prior knowledge lives in means and covariances instead of a buffer. On FewRel and TACRED, the method beats the rehearsal-free and prompt-based baselines, nearly matches rehearsal-based methods, and tops them on FewRel's final task.","feed_headline":"No stored data: prompt pools rival memory-based continual learning","feed_subtitle":"Per-task prompt pools plus synthetic representation replay match buffer-based baselines and beat FewRel's final task.","key_machinery":"The load-bearing mechanism is a task-specific prompt pool paired with per-relation Gaussian replay. Each pool $P_t$ contains $M$ prompts, each a single prefix expert with length $L_p=2$ and its own learnable key; the query vector $q(x)$ selects the top-$K$ prompts by cosine similarity (equation 13), and only those prompts and the shared classifier are trained for the current task. The design is justified by viewing prefix-tuning as inserting sparse experts into the mixture-of-experts structure of self-attention, so a separate key for each expert gives flexible selection and captures within-task variation while separate pools keep tasks apart. Forgetting in shared parameters is handled by the generative models: for each relation $r$, equations (16) and (17) fit Gaussians $\\mathcal{N}(\\mu^r_z,\\Sigma^r_z)$ to the prompted representations and $\\mathcal{N}(\\mu^r_q,\\Sigma^r_q)$ to the query vectors, and equations (18) and (19) train the task predictor $\\psi$ and relation classifier $\\phi$ on samples drawn from those Gaussians.","core_discovery":"The paper's central claim is that catastrophic forgetting in continual relation extraction can be controlled without storing any training instances by combining per-task prompt pools with generative replay of latent representations. WAVE-CRE freezes a BERT encoder, trains only the current task's prompt pool and the shared classifier head, and after each task fits a Gaussian per relation to the prompted representations $z^r=f_r(x^r_p)$ and another to the query vectors $q(x^r)$, following equations (16) and (17). Samples from these Gaussian distributions are then used in equations (18) and (19) to train the task predictor and the relation classifier, consolidating old tasks in the shared parameters. The experiments report that WAVE-CRE outperforms L2P, EPI, and HiDe-Prompt at every stage on both datasets, reaches a final accuracy of 85.0 on FewRel and 78.7 on TACRED, and on FewRel's last task exceeds the rehearsal-based CDec+ACA baseline.","pith_inferences":["If the per-relation Gaussian fit is adequate, the same representation-replay recipe should transfer to other continual text classification problems that use a frozen encoder, not just relation extraction.","The task predictor's per-relation output can be read as a soft assignment over relations; one could threshold its confidence to reject low-certainty inputs instead of always committing to a prompt pool.","A head-to-head comparison against a variant that replays the real stored representations would isolate exactly how much accuracy the bell-shaped approximation costs.","Because the encoder stays frozen, the method inherits whatever biases the off-the-shelf features carry; unfreezing the encoder would change the relation representation geometry and could break the fitted Gaussians."],"forward_implications":["Because only per-relation means and covariances are stored, the approach removes the storage and privacy costs of rehearsal buffers while keeping most of their accuracy.","The per-task prompt pool with a single expert per prompt keeps the learnable parameter count low (3.8M in the reported setup), so the method remains parameter-efficient as the number of tasks grows.","Training the task predictor on per-relation query distributions rather than on per-task aggregates improves task-identity precision, which removes the train/test prompt-selection mismatch that limits earlier prompt methods.","On the reported benchmarks the final accuracy lands close to the best rehearsal-based systems (85.0 on FewRel, 78.7 on TACRED), so the method is a practical substitute when buffers are undesirable."],"supporting_citations":[{"why":"Supplies the original prompt-pool design and the key-query selection rule (equation 13) that become task-specific in WAVE-CRE.","marker":"Wang et al. 2022b"},{"why":"Defines general and task-specific prompts, the rehearsal-free prompting paradigm WAVE-CRE builds on and compares against.","marker":"Wang et al. 2022a"},{"why":"Introduces hierarchical prompt decomposition and a task predictor; WAVE-CRE refines its per-task single prompt and per-task class predictor.","marker":"Wang et al. 2023a"},{"why":"Establishes the formal equivalence between prefix-tuning and adding sparse experts to mixture-of-experts models, motivating the prompt-pool-as-expert design.","marker":"Le et al. 2024a"},{"why":"Introduces deep generative replay, the paradigm WAVE-CRE adapts from raw data to latent representations.","marker":"Shin et al. 2017"},{"why":"Provides the consistent representation learning CRE baseline and the experimental protocol used for evaluation.","marker":"Zhao et al. 2022"},{"why":"Provides the RP-CRE rehearsal baseline and the TACRED task-split settings adopted in experiments.","marker":"Cui et al. 2021"},{"why":"Supplies the classifier-decomposition rehearsal baseline that WAVE-CRE compares against on both datasets.","marker":"Xia et al. 2023"}],"fun_headline_variants":["Data-free prompt pools rival memory buffers in continual RE","Prompt pools replace memory buffers for continual relation extraction","No buffers, no stored instances: prompt pools for continual relation extraction","Per-task prompt pools with generative replay: data-free continual RE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the encoded representations of each relation sit in one bell-shaped cluster, so a mean vector and covariance matrix can replace the real training examples; if a relation's examples split into several distinct groups or skew sharply, the replayed samples misrepresent it and the forgetting protection fails.","fun_headline_variants_meta":{"raw":{"variants":["Data-free prompt pools rival memory buffers in continual RE","Prompt pools replace memory buffers for continual relation extraction","No buffers, no stored instances: prompt pools for continual relation extraction","Per-task prompt pools with generative replay: data-free continual RE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.003025,"raw_usage":{"total_tokens":11444,"prompt_tokens":908,"completion_tokens":10536,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":10469}},"tokens_in":524,"tokens_out":10536,"duration_ms":73012,"temperature":1.0,"reasoning_tokens":10469,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:58:55.268184+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a relation whose encoded examples form two or more well-separated clusters, fit a two-component Gaussian mixture to the training representations from $D^r_z$. If the relation classifier trained on samples from the single fitted Gaussian loses more than a few points relative to one trained on the mixture, while the mixture version does not, then the single-Gaussian assumption is the weak link and the replay will misrepresent that relation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the consistent representation learning CRE baseline and the experimental protocol used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the RP-CRE rehearsal baseline and the TACRED task-split settings adopted in experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the classifier-decomposition rehearsal baseline that WAVE-CRE compares against on both datasets."}],"review_version":1}