{"id":"b1c2b6b5-ce18-46f4-8d2d-620117f08f15","arxiv_id":"2411.13097","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"SGLDL adds new labels to a label-distribution model by keeping old label relations fixed in a scalable graph and compensating new-label gradients, giving the best Euclidean distance on IMDB-WIKI age bins.","lead":"This paper defines incremental label distribution learning (ILDL), where a model must keep predicting distributions over old labels while new labels appear, and proposes a graph-based method, SGLDL, that freezes old label relationships and reweights gradients for new labels. It reports the best Euclidean distance at all ten checkpoints of an IMDB-WIKI age-distribution benchmark, but with one dataset, no error bars, and no code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed old-label preservation mechanism is not actually guaranteed: copying A^{t-1}_S in Eq. (2) does not freeze old-label relationships because SGCN's shared weights are updated on new tasks, and the paper's stationarity premise is untested.","rationale":"The paper presents a plausible algorithm and Table II consistently shows SGLDL at or near the top of the benchmark, which is real evidence. The reader's CONDITIONAL verdict is appropriate. My stress-test focuses on the mechanism that is supposed to justify the reported advantage. The load-bearing invariant is that the old block of the scalable correlation matrix can be copied unchanged (Eq. 2). This requires both an empirical stationarity of old-label relationships and an architectural guarantee that copying the adjacency actually preserves old-label output behavior. The first is asserted but not tested, and the second is contradicted by the GCN's shared-weight design: training on a new task updates the GCN weights, which are shared across old and new label nodes, so old classifiers change. Eq. 16's ℓRP is only a soft penalty, and no bound or measurement of drift is provided. The concrete test of measuring old-label classifier drift after each task would settle whether the central mechanism works. If the drift is large, the reported improvements must be attributed to other components (e.g., the loss reweighting or distillation), not to the claimed scalable graph invariant, and the paper's stated contributions would need substantial revision.","tokens_in":10711,"tokens_out":15007,"duration_ms":157630,"concrete_test":"Retrain SGLDL on IMDB-WIKI, and after each of the 10 tasks compute the cosine similarity between the old-label rows of SGCN output before and after the task (H^{t-1}_old vs H^t_old), and record the final ℓRP value. If the similarities are materially below 1 or if ℓRP is far from zero, the copied adjacency did not preserve old-label relationships, contradicting the mechanism. As a control, also run a version with the old GCN weights frozen; if it matches the reported SGLDL accuracy, the shared-weight updates are inconsequential, whereas if it differs, the claimed parameter locality is wrong.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that SGLDL preserves old-label performance by keeping inter-label relationships of old labels unchanged rests on the block copy A^{t-1}_S in Eq. (2). Two conditions must hold: (a) in the data, old-label relationships are truly invariant to the appearance of new labels, and (b) the model actually realizes that invariance. The paper asserts (a) in Section I but never tests it; on the IMDB-WIKI age benchmark it holds by construction (Gaussian age labels truncated and renormalized preserve ratios), so the experiment cannot validate the general ILDL claim. More seriously, (b) fails structurally: SGCN (Eq. 9) is a two-layer GCN with weights shared across all label nodes. Adding new labels and training on task t updates those shared weights, so the old rows of H^t are recomputed through a changed mapping even though the old adjacency block is copied. The claim in Fig. 3 that only new-label parameters are modified is therefore inconsistent with the equations; the relationship-preserving loss ℓRP (Eq. 16) is a soft regularizer, not a hard constraint, and no analysis bounds the drift. Consequently, the mechanism that is supposed to guarantee the benefit is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Incremental Label Distribution Learning (ILDL), where the label set grows over tasks and each training instance carries a distribution over all labels seen so far. It argues that the key difficulty is a 'label attention trap': adding new labels biases old-label predictions and causes naive retraining to rebuild old-label relationships that should be unchanged. The proposed SGLDL method combines a scalable correlation matrix (SCM) that extends an old adjacency block with new blocks, a two-layer GCN classifier (SGCN), and a new-label-aware gradient compensation loss, together with distillation and relationship-preserving losses. Experiments are reported on the IMDB-WIKI dataset for ten incremental tasks, with ablation of the three loss terms.","tokens_in":10981,"tokens_out":5523,"duration_ms":54459,"significance":"If the central mechanism could be established, ILDL would address a real gap: current LDL assumes a fixed label space, and the disease-diagnosis motivation is compelling. The SCM/SGCN design is an interesting way to make the classifier's label graph explicit and expandable, and the ablation studies target the right components. However, the paper's core claim that copying the old SCM block preserves old-label relationships is not supported by the model equations, and the experimental evidence is currently too thin to validate it. The contribution is potentially useful but needs substantial theoretical or architectural clarification and stronger empirical grounding.","major_comments":[{"comment":"Copying A^{t-1}_S into A^t_S in Eq. (2) does not preserve old-label relationships because SGCN's weights are shared across all label nodes in Eq. (9). When task t updates the network, the old rows H^t_j are recomputed through a changed GCN mapping even though the old adjacency block is unchanged; Fig. 3's statement that only new-label parameters are modified is therefore inconsistent with the equations. The relationship-preserving loss ℓRP in Eq. (16) is a soft regularizer, not a hard constraint, and no bound on old-node drift is given. The paper should either freeze the old-node computation, separate old/new GCN parameters, provide a formal drift guarantee, or directly measure old-relationship drift.","section":"Section III-B, Eq. (9)"},{"comment":"The stationarity premise that 'the inter-label relationship of old labels is not changed' is asserted but never tested. On IMDB-WIKI, the label distributions are Gaussian age labels; truncating and renormalizing the support when labels are added preserves the relative ratios by construction, so the benchmark cannot validate the assumption in settings where a new label can absorb probability mass from old labels. The authors should add a controlled experiment with constructed distribution shift, or a real dataset where new labels plausibly alter old-label correlations, and test whether the copied block remains accurate.","section":"Section I, Eq. (2)"},{"comment":"The experimental support is not yet sufficient. Only Euclidean distance is reported even though Table I defines four metrics; no standard deviations, seeds, or significance tests are given, and the 100-label row shows SGLDL at 0.551 versus 0.552 for BiC and 0.553 for SGLDL-w/oLRP, so the advantage is within the likely run-to-run variation. The Human Gene experiment promised in Section IV-B is also absent. The paper should report all metrics with multiple seeds and standard deviations, and either include the Human Gene results or remove the promise.","section":"Section IV-C, Table II"},{"comment":"The final loss weights λ1, λ2, and λ3 in Eq. (17) are never specified in Section IV, and no sensitivity analysis is provided. Because the method's benefit rests on balancing gradient compensation, distillation, and relationship preservation, the missing hyperparameters materially affect reproducibility and make it impossible to tell whether the result is robust or tuned. Please report the values and a sensitivity study.","section":"Section III-D, Eq. (17)"}],"minor_comments":[{"comment":"The contributions bullet says 'We tackle a real-world FL problem' but the paper is about ILDL, and the sentence 'which using a scalable label graph' is ungrammatical.","section":"Article contributions"},{"comment":"The notation for new labels in Eq. (13) assumes they are appended contiguously as indices c_{t-1}+1 through c_t, but the problem definition in Section II-A does not state this; please make the indexing explicit.","section":"Section II-A, Eq. (13)"},{"comment":"The formulas for e_{ij}^k and r_{ij}^k are difficult to parse because of the radical/division structure and the conditions in parentheses; please rewrite them unambiguously.","section":"Section III-A, Eqs. (5) and (7)"},{"comment":"The GloVe embedding citation appears unresolved as '[cite]' and must be completed.","section":"Section III-B"},{"comment":"Table II contains typographical artifacts such as '0. 372', '.281', and '.635'; please reformat the table.","section":"Section IV-C, Table II"}],"recommendation":"major_revision","confidential_remarks":"The central mechanism needs to be fixed: the parameter-update claim in Fig. 3 is contradicted by the shared-weight GCN in Eq. (9), and the soft regularizer in Eq. (16) does not provide the claimed guarantee. If the authors can make the old-label preservation claim true by design or prove a drift bound, and provide proper multi-seed, multi-metric experiments, the paper could become publishable. I also note that the citation for 'label attention trap' [5] appears unrelated to the term as used in this manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper earns credit for defining Incremental Label Distribution Learning as a real problem and for a concrete mechanism: an expandable correlation matrix with a frozen old block, a new-label-aware gradient compensation loss, and a relationship-preserving regularizer. That combination is not in the cited literature. The ablation study shows each loss contributes, and Table II ranks SGLDL first at all ten checkpoints on IMDB-WIKI. This is a legitimate start.\n\nThe soft spots are substantial, though. The stress-test note lands: copying A^{t-1}_S into A^t_S does not freeze old-label relationships, because SGCN's shared weights are updated while training on new labels, so old rows of H^t are recomputed through a changed mapping. The relationship-preserving loss is a soft regularizer, not a constraint, and no analysis bounds the drift. The paper's stationary premise—that old-label relations never change—is asserted but never tested. Worse, the IMDB-WIKI age benchmark makes that premise true by construction, since the label distributions are truncated and renormalized Gaussians that preserve ratios. So the main experiment cannot validate the general ILDL claim.\n\nThe empirical evidence is also too thin: one dataset, one metric, no standard deviations or seeds, no code, and the promised Human Gene experiment is missing. The reported margins at later checkpoints are tiny (0.551 vs 0.552 at 100 labels). There are also signs of incomplete polish: placeholder citations, undefined notation in the correlation formulas, a likely typo in the softmax denominator in Eq. (11), and no reported values for lambda_1 through lambda_3.\n\nNone of this proves the method is wrong. The problem is important, the design is sensible, and the flaws are fixable. But the central claim about preserving old-label relationships is not yet supported, and the experiments need to be broader and more rigorous.\n\nWho should read it: anyone working on continual LDL or label-space growth, particularly for medical or age-estimation applications. It deserves a serious referee, but the referee should push hard on the mechanism and ask for more evidence. If I were the editor, I would send it to review, not desk reject—but I would expect heavy revision.","headline":"A genuinely new ILDL problem framing and a plausible method, but the old-label preservation guarantee is not established by the equations and the experiments are too thin to carry it.","tokens_in":11486,"tokens_out":1444,"would_cite":false,"duration_ms":17194,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces Incremental Label Distribution Learning (ILDL) and a scalable graph method, SGLDL, that keeps old-label relationships fixed while adding new labels, and it reports the best Euclidean-distance results on IMDB-WIKI age…","keywords":["incremental label distribution learning","label attention trap","scalable label graph","graph convolutional network","label distribution learning","class-incremental learning","gradient compensation loss"],"falsifier":"Split the same data into two ILDL sequences: in one the new label is independent of old labels, in the other the new label is engineered to change old-label co-occurrence statistics; if SGLDL's Euclidean distance on old labels degrades in the second sequence while a from-scratch relearned model does not, the frozen old block is responsible.","tokens_in":10497,"feed_emoji":"🏷️","tokens_out":13248,"duration_ms":113431,"temperature":0.7,"pith_summary":"Label distribution learning (LDL) outputs a distribution over labels, but standard LDL assumes a fixed label set. This paper introduces Incremental Label Distribution Learning (ILDL), where new labels appear over time, and identifies the 'label attention trap': when the output vector lengthens, old-label probabilities shrink and the model re-learns old labels instead of focusing on new ones. The paper argues that the inter-label relationships among old labels do not change, so the right strategy is to keep them fixed and add only the new relations, encoded in a scalable correlation matrix. It proposes SGLDL, which combines that scalable label graph with a new-label-aware gradient compensation loss, and reports better Euclidean-age-distance results than CIL-based combinations on the IMDB-WIKI benchmark.","feed_headline":"Frozen label relations let models add labels without relearning","feed_subtitle":"SGLDL freezes old-label relations and reweights gradients, beating retraining baselines as labels grow from 10 to 100.","key_machinery":"The load-bearing mechanism is the scalable correlation matrix (SCM), a block-structured label graph $A_S^t = \\begin{pmatrix} A_S^{t-1} & E^t \\\\ R^t & M^t \\end{pmatrix}$ whose upper-left block is the unchanged old-label relation matrix; the new blocks are estimated from co-occurrence statistics as coefficients of variation. The label graph makes inter-label relations explicit and growable: labels are graph nodes initialized with pretrained word embeddings, and a two-layer graph convolutional network (SGCN) maps the graph to label-specific classifiers, so a new label is one new node rather than a fully relearned classifier. The other load-bearing piece is the new-label-aware gradient compensation loss $\\ell_{NC}$, which normalizes old and new label gradients separately and amplifies new-label gradients in the Canberra-distance prediction loss; a distillation loss $\\ell_{DT}$ and a relation-preserving loss $\\ell_{RP}$ limit forgetting of old outputs and old graph embeddings.","core_discovery":"The central claim is that incremental label distribution learning can avoid the label attention trap by separating inter-label relations from feature extraction and freezing the old relations. Concretely, SGLDL stores old-label relations in the upper-left block of a scalable correlation matrix $A_S^t = \\begin{pmatrix} A_S^{t-1} & E^t \\\\ R^t & M^t \\end{pmatrix}$, copies that block unchanged when new labels arrive, and computes only the new-to-new, old-to-new, and new-to-old blocks. A two-layer graph convolutional classifier consumes this matrix, and a new-label-aware gradient compensation loss re-weights gradients so that new labels learn faster while old labels are protected by distillation and relation-preserving losses. On IMDB-WIKI age estimation, split into ten incremental tasks, SGLDL achieves the best Euclidean-distance errors among the tested combinations of Adam-LDL-SCL, iCaRL, BiC, and SS-IL, and the ablations show every loss component contributes.","pith_inferences":["Editorial extension: if a real domain violates the frozen-relation assumption, the natural repair is to detect drift in old-label co-occurrence statistics and refresh only the affected block, preserving the graph's sparsity.","Editorial extension: the gradient compensation loss is not tied to age estimation and should transfer to other growing-label taxonomies, such as medical codes or product tags; a test on such a dataset would show how far the IMDB-WIKI result generalizes.","Editorial extension: the paper's coefficient-of-variation construction yields a direct diagnostic — if the ratio statistics among old labels change when a new label arrives, the fixed-relation assumption is violated and the frozen block should be updated."],"forward_implications":["An LDL model can accept a new label without retraining all old-label parameters: only the graph rows and columns connecting the new label to the existing labels must be learned.","Sequences of tasks with growing label sets can be handled as long as every task's training distribution contains all labels the model has seen, which is the ILDL setting defined in the paper.","The new-label-aware gradient compensation loss, together with distillation and relation-preserving losses, should reduce both underfitting of new labels and catastrophic forgetting of old labels.","Because the classifier is a GCN over a scalable label graph, parameter growth when labels are added is concentrated near the new label instead of spread over the full classifier.","The IMDB-WIKI experiments show a dedicated ILDL method can beat generic CIL-plus-LDL combinations, and the ablations attribute the gain to all three loss terms."],"supporting_citations":[{"why":"Defines label distribution learning, the problem setting that ILDL extends to a growing label space.","marker":"[1]"},{"why":"Provides Adam-LDL-SCL, the state-of-the-art LDL method used as a standalone baseline and as the base learner combined with three class-incremental methods.","marker":"[4]"},{"why":"Supplies the 'label attention trap' notion that the paper formalizes as the central barrier to incremental LDL.","marker":"[5]"},{"why":"Introduces graph-based label distribution learning, the direct precursor for representing inter-label relations as a graph.","marker":"[10]"},{"why":"Inspires the distillation loss used to preserve old-label knowledge during incremental tasks.","marker":"[12]"},{"why":"Supplies the IMDB-WIKI age dataset with Gaussian-smoothed age labels used in all experiments.","marker":"[13]"},{"why":"iCaRL is a class-incremental baseline that SGLDL is compared against when combined with Adam-LDL-SCL.","marker":"[14]"},{"why":"BiC is a class-incremental baseline whose validation-set bias correction makes it the strongest CIL comparison.","marker":"[15]"},{"why":"SS-IL is a class-incremental baseline with separated softmax and task-specific distillation that SGLDL is compared against.","marker":"[16]"},{"why":"Supplies DLDL, the label distribution learning method used atop the image backbone for the IMDB-WIKI experiments.","marker":"[23]"}],"fun_headline_variants":["Freeze old label links to add new ones fast","Add labels without relearning old ones","Graph trick lets models grow label sets incrementally","New-label-aware gradients beat full retraining","Frozen graph blocks let models add labels incrementally"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's load-bearing premise is that adding a new label does not change the relationships among old labels, so the old part of the label graph can be copied forward; if a new label changes old-label relationships, the copied block introduces systematic error.","fun_headline_variants_meta":{"raw":{"variants":["Freeze old label links to add new ones fast","Add labels without relearning old ones","Graph trick lets models grow label sets incrementally","New-label-aware gradients beat full retraining","Frozen graph blocks let models add labels incrementally"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000435,"raw_usage":{"total_tokens":2241,"prompt_tokens":995,"completion_tokens":1246,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":1176}},"tokens_in":611,"tokens_out":1246,"duration_ms":8261,"temperature":1.0,"reasoning_tokens":1176,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:51:06.811481+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Split the same data into two ILDL sequences: in one the new label is independent of old labels, in the other the new label is engineered to change old-label co-occurrence statistics; if SGLDL's Euclidean distance on old labels degrades in the second sequence while a from-scratch relearned model does not, the frozen old block is responsible.","supporting_citations":[{"cited_title":"Label distribution learning with label correlations on local samples,","cited_arxiv_id":null,"evidence_quote":"Provides Adam-LDL-SCL, the state-of-the-art LDL method used as a standalone baseline and as the base learner combined with three class-incremental methods."},{"cited_title":"Few-shot charge prediction with discriminative legal attributes,","cited_arxiv_id":null,"evidence_quote":"Supplies the 'label attention trap' notion that the paper formalizes as the central barrier to incremental LDL."},{"cited_title":"Gldl: Graph label distribution learn- ing,","cited_arxiv_id":null,"evidence_quote":"Introduces graph-based label distribution learning, the direct precursor for representing inter-label relations as a graph."},{"cited_title":"Lifelong learning via progressive distillation and retrospection,","cited_arxiv_id":null,"evidence_quote":"Inspires the distillation loss used to preserve old-label knowledge during incremental tasks."},{"cited_title":"Deep expectation of real and apparent age from a single image without facial landmarks,","cited_arxiv_id":null,"evidence_quote":"Supplies the IMDB-WIKI age dataset with Gaussian-smoothed age labels used in all experiments."},{"cited_title":"Ss- il: Separated softmax for incremental learning,","cited_arxiv_id":null,"evidence_quote":"SS-IL is a class-incremental baseline with separated softmax and task-specific distillation that SGLDL is compared against."}],"review_version":1}