{"id":"d5c2cf73-160f-4b14-9448-550450009b5f","arxiv_id":"2505.12526","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Using historical labels as pseudo-targets lets temporal GNNs train on label-scarce batches, cutting convergence time by up to 15x on TGB datasets, but the theoretical proof and SOTA claims are weakly supported.","lead":"Temporal graph networks often wait for labeled events to update, so most batches train nothing. This paper proposes feeding back past labels as pseudo-targets to train every batch, reporting large speedups in wall-clock convergence on Temporal Graph Benchmark datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Speedup claim rests on a nonstandard 5% training split; standard TGB protocol may not show any acceleration.","rationale":"The theoretical analysis has clear gaps: Lemma 2's product model does not match the actual historical average, and the gradient-variance proportionality is asserted rather than derived. However, even a corrected label-variance calculation gives a variance that decreases with history length, so the theoretical direction may survive revision. The empirical claim, by contrast, is evaluated only under a self-selected 5% truncation, and the paper's own justification for the truncation (full training sets cause convergence within one epoch) predicts that the speedup will not transfer to the standard benchmark. This is the least secure condition for the central claim as stated in the abstract. I therefore focus the concrete test on the standard TGB protocol. The reader's verdict of REJECT remains appropriate, but for a reason closer to the experimental protocol than to the label-model mismatch in Lemma 2. If the full-split test shows no speedup, the paper should be revised to explicitly scope the claim to truncated or label-sparse training regimes and to remove the unsupported 'new SOTA' assertion.","tokens_in":14023,"tokens_out":18356,"duration_ms":210183,"concrete_test":"Run the same TGNv2 experiments on the standard full TGB training splits (70% chronological, no 5% truncation) for tgbn-genre, tgbn-reddit, and tgbn-token, with identical hyperparameters, early stopping on validation, at least 3 seeds, and a fixed wall-clock budget. Compare time-to-best-validation and final NDCG@10 for Default versus HA/MA/PF. If pseudo-label methods do not reach best validation faster than Default within one epoch or within a fixed time budget, the 15x claim is an artifact of the modified protocol. Additionally, report the official TGNv2 NDCG@10 numbers for these datasets so the 'new SOTA' assertion can be checked against published baselines.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 (Table 1) states that for tgbn-genre, tgbn-reddit, and tgbn-token the authors retain only the last 5% of chronologically ordered training edges, explicitly because on the full training sets models converge within a single epoch, which would obscure the effect of pseudo-labeling. All speedup numbers in Table 2 for those datasets are therefore not measured on the standard TGB benchmark. The abstract's unqualified 'up to 15x' and 'new SOTA' claims require evaluation on the standard 70/15/15 splits. This is load-bearing because the headline empirical claim is conditional on an artificial regime: the Default baseline is forced to take many epochs on the truncated data, while HA/MA converge in one epoch for genre and token and two epochs for reddit. On the full training sets, where Default already converges within one epoch, the pseudo-label advantage may vanish or reverse. No error bars or multiple seeds are reported, and no comparison to published TGB results is provided, so the 'new SOTA' statement in the metadata abstract is unsupported. If the full-split speedup is absent, the central claim of the paper as stated is not established.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to train temporal graph networks on every batch by using pseudo-labels computed from past observed targets, with three aggregation strategies: Historical Average (HA), Moving Average (MA), and Persistent Forecast (PF). It claims a theoretical result that historical averaging reduces gradient variance and accelerates convergence by a factor of min(h,k), where h is the history length and k the number of classes, and it reports experiments on four Temporal Graph Benchmark datasets showing up to 15x faster training for TGNv2 with competitive accuracy. The method introduces no trainable parameters and is intended to be architecture-agnostic. A notable feature of the experimental protocol is that for three of the four datasets only the last 5% of the chronologically ordered training edges are retained, which the authors justify by saying that on the full training sets models converge within a single epoch.","tokens_in":14214,"tokens_out":11141,"duration_ms":117719,"significance":"If the claims were correct, the paper would offer a simple, zero-parameter, architecture-agnostic technique for mitigating sparse supervision in temporal GNNs, and the ablation studies on edge-order and target-order shuffling are useful. The appendix contains complete proofs of the stated lemmas and theorems, and the method description is mostly clear. However, the theoretical derivation rests on a label model that does not match the implemented averaging process and on an unproved proportionality between label variance and gradient variance; moreover, the headline speedups are measured on a modified benchmark with a nonstandard 5% training split and no comparison to standard TGB results. These issues undermine the central claims of the paper as stated, so the contribution is not established in its current form.","major_comments":[{"comment":"The random variable t_h = eta_k * xi, with eta_k independent of xi, does not model the historical average of one-hot labels under missingness. For h independent observations in which a fixed label is observed with probability u/k per observation, the component average is Binomial(h, u/k)/h, whose variance is u(k-u)/(k^2 h) and vanishes as h grows. The paper's formula u(k-1)/(k^2 h) + u(1-u)/k^2 retains an h-independent term u(1-u)/k^2, so the claimed min(h,k) speedup in Theorem 3 follows from a label model that the method does not implement. The theorem therefore does not establish the stated convergence acceleration; the variance-reduction result must be re-derived for the actual averaging process, and with the correct model the speedup factor would be h rather than min(h,k).","section":"Section 3.2, Lemma 2 (Appendix A, Lemma 4)"},{"comment":"The assertion that the gradient variance can be written as \\tilde{c} Var(t_h) for a non-stochastic constant \\tilde{c} across all layers is not justified. For the last layer, Var(p_i - y_i) is not equal to Var(y_i) in general; it also depends on the variance of the model output p_i through the joint distribution of inputs and labels. For earlier layers, the chain rule introduces products of activations and Jacobians that are random and correlated with the label, so a label-independent constant \\tilde{c} is not available without additional assumptions. Since Theorem 3 uses this proportionality to convert the label variance into a gradient-variance bound, the convergence-rate claim is not supported by the provided derivation.","section":"Section 3.2, paragraph after Lemma 2"},{"comment":"The reported speedups for tgbn-genre, tgbn-reddit, and tgbn-token are obtained on a modified protocol that keeps only the last 5% of the chronologically ordered training edges, not on the standard TGB 70/15/15 split. The paper explicitly states that on the full training sets models converge within one epoch, which would obscure the effect of pseudo-labeling. Consequently, the abstract's unqualified claims of up to 15x acceleration and 'new SOTA' are not established for the standard benchmark; the experiments compare methods only in an artificial low-data regime. No results on the full training splits, no comparison with published TGB numbers, and no error bars or multiple-seed statistics are reported, so even within the modified regime the quantitative claims are not verified.","section":"Section 5.1, Table 1; Section 5.2, Table 2"},{"comment":"The manuscript is internally inconsistent about the training target. The supervised-batch description says predictions are compared to ground-truth labels y_t^{(v)}, but Eq. (6) states that the loss replaces y_t^{(v)} with pseudo-labels \\tilde{y}_t^{(v)} 'for almost all nodes and batches.' It is therefore unclear whether supervised batches use true labels, pseudo-labels, or a mixture; this determines whether the pseudo-label gradient is unbiased and is essential for reproducing the experiments. The method definition must be clarified before the empirical results can be interpreted.","section":"Section 4.1, Eqs. (5)-(6); Section 4.2"}],"minor_comments":[{"comment":"The metadata abstract refers to 'Moving-Averaged Labels (MAL)' and to experiments on TGNv2 and DyRepv2, while the full-text abstract refers to 'History-Averaged Labels (HAL)' and the experiments cover only TGNv2; these inconsistencies should be corrected.","section":"Abstract and title"},{"comment":"The paper says the method has zero additional parameters, but the noise scale gamma, the noise amplitude alpha, the moving-average window w, and the 5% truncation threshold are tunable hyperparameters; their selection procedure and sensitivity are not reported.","section":"Section 4.2"},{"comment":"The 'N steps' metric in Table 2 is not defined precisely; it should state whether it counts gradient updates, batches, or forward passes.","section":"Section 5.2"},{"comment":"The logarithmic time axis makes the claimed speedup difficult to read; the figure would benefit from marking the convergence point for each method and from showing the Default curve beyond the early training range.","section":"Figure 3"}],"recommendation":"reject","confidential_remarks":"The paper's advertised claims are considerably stronger than what the experiments and theory support. The nonstandard 5% training split, the misspecified label model in Lemma 2, and the unproved gradient-variance proportionality are load-bearing issues, and the method description is internally inconsistent about the training target. In my view a substantially revised version could be reconsidered, but the current manuscript does not establish the central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the short version: the idea is worth a look, but the proof is wrong and the headline speedups come from a nonstandard training split that the abstract glosses. I wouldn't accept this as is; I'd ask for major revision.\n\nWhat's new: applying simple historical pseudo-labels—historical average, moving average, persistent forecast—to make every batch trainable for temporal GNNs looks new relative to the cited literature, and it's a zero-parameter, architecture-agnostic trick. Table 2 shows consistent step and wall-clock reductions across four datasets in their setup, with test NDCG roughly matching the default baseline on most. That internal consistency deserves credit.\n\nThe soft spots are substantive. First, Lemma 2 doesn't describe the actual empirical frequency of a label. Under their own assumptions—u chance of an observation per time step, uniform over k labels—the count for any label is Binomial(h, u/k), so the empirical frequency has variance (u/k)(1−u/k)/h. That's a 1/h reduction, with no extra u(1−u)/k^2 term. Their product model with independent Bernoulli and Binomial produces that spurious term, which doesn't vanish as h grows, and it's what generates the min(h,k) speedup claim. So the central theoretical result is incorrect. Second, the chain from label variance to gradient variance to regret is asserted via 'chain rule' and a constant c, not derived. Even a fixed label variance wouldn't justify the regret bound without more steps. Third, the experimental protocol: for tgbn-genre, reddit, and token, training is truncated to the last 5% of edges because on full data models converge in one epoch. The speedup numbers are therefore not measured on the standard TGB splits, and the abstract's unqualified 'up to 15x' and 'new SOTA' claims rest on this modified regime. There are no error bars or multiple seeds, and no comparison to published TGB results, so 'new SOTA' is unsupported.\n\nThat said, the within-regime result is plausible: the aggregation methods do reach the best validation NDCG in far fewer steps and seconds than Default, and the window-size and shuffling ablations are informative. The paper is aimed at practitioners in recommender systems and fraud detection who care about time-to-accuracy, and at researchers studying label-augmentation tricks.\n\nMy call: send it to peer review—the idea is plausible and the experiments are extensive—but expect heavy revision. As written, the theory and benchmark claims would not survive close scrutiny. A revision with a corrected label model, a rigorous gradient-variance argument, standard TGB splits, and seeds would be a genuinely useful contribution.","headline":"A plausible pseudo-labeling trick for temporal GNNs, but the variance proof is flawed and the headline speedups rely on a nonstandard 5% training split.","tokens_in":14757,"tokens_out":4438,"would_cite":false,"duration_ms":44129,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"History-averaged pseudo-labels reduce gradient variance in temporal GNN training, cutting time-to-accuracy by up to 15 times on standard benchmarks.","keywords":["temporal graph networks","pseudo-labeling","label sparsity","gradient variance reduction","convergence acceleration","dynamic graph learning","moving average labels","Temporal Graph Benchmark"],"falsifier":"Train the method on a synthetic temporal graph where each node's true label is redrawn uniformly at random at every timestamp, so historical averages carry no information about the current label; if convergence is still faster than vanilla SGD, the reported speedup is not explained by unbiased variance reduction.","tokens_in":13763,"feed_emoji":"⏱","tokens_out":10210,"duration_ms":93030,"temperature":0.7,"pith_summary":"In temporal graphs, most training batches arrive without labels, so models update memory but do not learn. This paper claims that such gaps can be filled with cheap pseudo-targets: for each node, average (or exponentially average, or copy) the labels observed in that node's past, and train on those targets as if they were real. The central theoretical assertion is that this aggregation lowers the variance of stochastic gradients, and that under slowly changing preferences the convergence speed improves by a factor of order $\\min(h,k)$, where $h$ is how much history is aggregated and $k$ is the number of classes. Empirically, using the TGNv2 model on the four Temporal Graph Benchmark datasets, the paper reports reaching top validation performance up to 15 times faster while keeping accuracy competitive. The appeal is that the method adds no parameters, leaves the loss and architecture untouched, and turns previously idle batches into productive updates.","feed_headline":"Historical labels cut temporal GNN training time up to 15x","feed_subtitle":"Pseudo-targets from past interactions fill unlabeled batches, cutting time-to-accuracy on temporal graph benchmarks.","key_machinery":"The central object is the history-averaged pseudo-label $\\tilde{y}_t^{(v)}$, an aggregate of all supervision signals observed for node $v$ before time $t$; the abstract calls this Moving-Averaged Labels (MAL), the body calls it History-Averaged Labels (HAL). Three variants are used: Historical Average (plain mean of all past targets), Moving Average (exponentially weighted recent labels), and Persistent Forecast (last observed label). The pseudo-label enters the objective by replacing $y$ in the cross-entropy loss while the model, loss, and architecture stay unchanged. The theoretical load is carried by Lemma 2, which models one component of the pseudo-label as the product of an observation-indicator Bernoulli $\\xi$ and a scaled multinomial frequency $\\eta_k$, derives the variance $\\mathrm{Var}(t_h)=u(k-1)/(k^2 h)+u(1-u)/k^2$, and feeds it into the SGD regret bound of Theorem 1 to produce the $\\min(h,k)$ speedup factor.","core_discovery":"The paper's discovery, stated in good faith, is that the slow drift of node preferences is itself a usable supervisory signal. When ground-truth targets are missing, replacing them with the node's historical label distribution (computed as a historical average, an exponential moving average, or the last observed value) gives an unbiased estimator of the current target, and the extra averaging shrinks the per-label variance from order $1/k$ to order $1/\\min(h,k)$. Substituting this variance estimate into the standard SGD regret bound yields an upper bound on regret that is smaller by the same factor, which the paper reads as a proof that pseudo-labels accelerate convergence. On the TGB suite, TGNv2 with these pseudo-targets reaches its best validation NDCG@10 in fewer steps and less wall-clock time, up to 15 times faster on the harder datasets, with test quality on par with or slightly better than the vanilla model.","pith_inferences":["Outside the paper's experiments, the same pseudo-supervision logic could apply to link prediction and temporal knowledge graph completion, where supervision is also sparse and sequential.","A testable extension is to make the aggregation window adaptive: choose per-node weighting from validation performance, which could handle both stable and volatile users without adding parameters.","The paper's own target-shuffling ablation points to a boundary: when preferences drift fast, persistent forecast degrades while moving average retains more quality, so a drift-aware mixture of the three variants is a natural next step.","If the variance argument transfers to other architectures, label-level variance reduction would complement optimizer-level methods such as Adam or variance-reduced SGD, attacking a different component of optimization noise."],"forward_implications":["On label-sparse temporal graphs, training becomes continuous: batches without ground truth still produce gradient updates instead of being skipped.","Time-to-accuracy shrinks by up to an order of magnitude on the tested benchmarks, which matters when models must be retrained frequently on streaming data.","Because the method adds no parameters and changes neither the loss nor the architecture, it can be dropped into any temporal GNN, not just TGNv2.","The three aggregation variants provide a stability-versus-reactivity dial: persistent forecast suits stable preferences, moving average suits drifting ones, and historical average suits long-run baselines.","The theoretical factor $\\min(h,k)$ predicts diminishing returns from extending history once history length exceeds the number of target categories."],"supporting_citations":[{"why":"Defines the Temporal Graph Network architecture with memory and message passing that this paper trains.","marker":"[24]"},{"why":"TGNv2, the backbone model and strongest baseline on the Temporal Graph Benchmark that the pseudo-label method is applied to and compared against.","marker":"[30]"},{"why":"The Temporal Graph Benchmark supplying the four dynamic node property prediction datasets, the chronological splits, and the evaluation protocol.","marker":"[11]"},{"why":"Provides the SGD regret bound (Theorem 1) into which the paper substitutes its variance estimate to derive the convergence speedup.","marker":"[27]"},{"why":"Classical treatment of SGD convergence showing regret proportional to gradient variance, which the theoretical analysis builds on.","marker":"[26]"},{"why":"Models online interactions as a temporal point process driven by a slowly changing latent state, motivating the temporal-consistency assumption behind historical pseudo-labels.","marker":"[5]"},{"why":"Analysis suggesting that for some sequential datasets event order can be ignored, supporting the use of historical label aggregates.","marker":"[16]"}],"fun_headline_variants":["Pseudo-labels from history speed temporal GNN training up to 15x","Adaptive pseudo-supervision cuts temporal GNN training time by 15x","Moving-averaged labels fill gaps, making temporal GNNs train 15x faster","Historical label distributions accelerate temporal graph learning up to 15x","Soft labels from past events reduce variance, yielding 15x faster temporal GNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on historical pseudo-labels being an unbiased proxy for a node's current label and on label variance being proportional to gradient variance; if past preferences stop tracking current ones, or if label noise does not translate linearly into gradient noise, the claimed convergence speedup does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Pseudo-labels from history speed temporal GNN training up to 15x","Adaptive pseudo-supervision cuts temporal GNN training time by 15x","Moving-averaged labels fill gaps, making temporal GNNs train 15x faster","Historical label distributions accelerate temporal graph learning up to 15x","Soft labels from past events reduce variance, yielding 15x faster temporal GNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000248,"raw_usage":{"total_tokens":1527,"prompt_tokens":909,"completion_tokens":618,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":513}},"tokens_in":525,"tokens_out":618,"duration_ms":5850,"temperature":1.0,"reasoning_tokens":513,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:32:01.552390+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the method on a synthetic temporal graph where each node's true label is redrawn uniformly at random at every timestamp, so historical averages carry no information about the current label; if convergence is still faster than vanilla SGD, the reported speedup is not explained by unbiased variance reduction.","supporting_citations":[{"cited_title":"Huang, F","cited_arxiv_id":null,"evidence_quote":"The Temporal Graph Benchmark supplying the four dynamic node property prediction datasets, the chronological splits, and the evaluation protocol."},{"cited_title":"Shamir and T","cited_arxiv_id":null,"evidence_quote":"Provides the SGD regret bound (Theorem 1) into which the paper substitutes its variance estimate to derive the convergence speedup."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Models online interactions as a temporal point process driven by a slowly changing latent state, motivating the temporal-consistency assumption behind historical pseudo-labels."},{"cited_title":"Klenitskiy, A","cited_arxiv_id":null,"evidence_quote":"Analysis suggesting that for some sequential datasets event order can be ignored, supporting the use of historical label aggregates."}],"review_version":1}