{"id":"276bfac8-edb0-4102-8c40-ff8044dd1506","arxiv_id":"1908.02831","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new multislice-PHATE visualization reveals training dynamics, catastrophic forgetting, and a link between hidden-unit heterogeneity and generalization in neural networks.","lead":"This paper introduces M-PHATE, a visualization method that shows how a neural network's internal representations change during training. It could help engineers see why some networks forget old tasks or generalize better, without needing held-out data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative headlines—Table 1's 'best on all measures' and Table 3's ρ=-0.98—rest on a single training run and on metrics computed from the PHATE embedding, not directly from activations; replicate with multiple seeds and compare against direct activation-space variance before accepting them.","rationale":"The reader's verdict already flags thin quantitative evidence and undefined metrics, so this stress test supports the CONDITIONAL verdict rather than changing it. The paper's core methodological idea is coherent: the multislice kernel is described with parameters (k=2, α=5, κ=25 in S6), and the per-epoch z-score is defensible because an affine reparameterization of a hidden unit can be absorbed into the following layer. The supplemental repeat (S2) is a useful check that training-data-only inputs give qualitatively similar results, though the variance values differ substantially (e.g., Dropout 382 vs 59). The remaining gap is that the key quantitative claims are load-bearing for the central narrative but are not supported by a replicated protocol or by metrics computed directly from activations. That is a reason to require more evidence before full acceptance, not a reason to reject the paper. If the proposed reproducibility test passes, the concern is resolved; if it fails, the quantitative claims should be removed or substantially weakened.","tokens_in":12561,"tokens_out":11324,"duration_ms":129890,"concrete_test":"Run each of the eight conditions in Section 4.4 with at least 10 random seeds and identical hyperparameters. For each run, compute (a) the paper's summed PHATE-embedding variance and (b) the same variance on the z-scored activation tensor T before embedding, e.g., mean over epochs of the average pairwise squared distance among hidden-unit activation vectors. Then report the Spearman correlation of each measure with memorization error across conditions and seeds. If the correlation is not robustly negative for both measures, or if the embedding-based measure disagrees with the activation-based measure, the claimed ρ=-0.98 is an artifact of the embedding procedure rather than a property of the networks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing point is not the M-PHATE construction itself, which is clearly specified, but the quantitative evidence offered for its advantages and for the heterogeneity-generalization finding. Table 1 is based on one network trained for 300 epochs (Section 4.1); no seeds, error bars, or ablations are reported, so the claimed ordering over ISOMAP and t-SNE may be single-run noise. Table 3's ρ=-0.98 correlation uses 'summed variance per epoch of the PHATE visualization' as the measure of heterogeneity. That quantity is evaluated after PHATE's MDS step with n_landmark=3000 and an automatically selected diffusion time t; it therefore mixes true unit heterogeneity with the embedding's scale and, potentially, landmark-subsample variability. The same concern applies to the ARI in Section 4.3, where the clustering algorithm applied to the embedding is not specified. These issues do not falsify the method, but they leave the paper's quantitative claims not yet separated from embedding artifacts or single-configuration luck.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Multislice PHATE (M-PHATE), a visualization method that combines a multislice kernel over hidden-unit activations at different training epochs with the PHATE dimensionality-reduction algorithm. The authors claim this is the first method designed explicitly to visualize how a network's hidden representations evolve during training, and they evaluate it against PHATE, diffusion maps, Isomap, and t-SNE in both standard and multislice variants. Two application vignettes are presented: continual learning, where the visualizations are claimed to reveal catastrophic forgetting, and generalization, where an embedding-derived heterogeneity measure is reported to correlate strongly (rho = -0.98) with memorization error. The paper includes an implementation on GitHub and a supplementary section that repeats the generalization experiment using training data instead of validation data.","tokens_in":12791,"tokens_out":10031,"duration_ms":95559,"significance":"If the quantitative claims hold, M-PHATE would be a useful interpretability tool that connects training dynamics to network performance, and the paper's open-source implementation and the supplementary training-data replication are commendable strengths. The algorithmic construction is clearly specified and reproducible in principle. However, the central quantitative evidence is currently fragile: the headline comparisons rest on a single training run without error bars, one of the key metrics is undefined, the reported correlation in the continual-learning experiment is not reproducible from the tabulated values, and the generalization correlation is computed from the PHATE embedding rather than from raw activations. These issues do not invalidate the method but leave the paper's strongest claims not yet separated from single-configuration luck or embedding artifacts.","major_comments":[{"comment":"The neighborhood-preservation comparison is based on a single network trained for 300 epochs, with no seeds, error bars, or significance tests. The text also defines a 'Loss Correlation' as the Spearman correlation between the rate of change of each hidden unit and the rate of change of the validation loss, but 'rate of change' is never defined: it is unclear whether this is a per-epoch displacement in the embedding, a difference in activation vectors, or something else. Additionally, the interslice neighborhood-preservation metric is evaluated against the same temporal distances T(:,i) that are explicitly encoded in the M-PHATE interslice kernel, so high scores for M-PHATE partly reflect the kernel construction rather than an independent ground truth. Please report multiple seeds with error bars, define the rate-of-change and loss-correlation metric, and discuss or control for the circularity in the interslice metric.","section":"§4.2, Table 1"},{"comment":"The Adjusted Rand Index experiment is not reproducible as described. The clustering algorithm applied to the visualization is not specified; the text only mentions '6 different choices of clustering parameter (between 3–8 clusters)'. More importantly, the text claims that the average ARI is strongly negatively correlated with final validation loss (rho = 0.94), but this is internally inconsistent: a negative correlation would have rho < 0, and a Spearman rank correlation over the nine entries in Table 2 gives approximately -0.73, not 0.94. Within the task-learning columns, Adam has the lowest validation loss but the lowest ARI, contradicting the claimed monotone relationship. Please specify the clustering method, report the exact correlation computation, and reconcile the text with the tabulated values.","section":"§4.3, Table 2"},{"comment":"The headline finding that heterogeneity correlates with generalization (rho = -0.98) uses 'summed variance per epoch of the PHATE visualization' as the measure of heterogeneity. This quantity is computed after PHATE's MDS step with n_landmark = 3000 and an automatically selected diffusion time t, so it can reflect the embedding scale and landmark subsampling rather than any intrinsic property of the hidden units. The supplementary replication with training data is a step in the right direction, but it uses the same embedding pipeline. Please compute the variance directly in the z-scored activation space (or in the multislike kernel space) and show that the correlation is robust to the embedding parameters, or clearly state that the claim concerns the visualization-derived quantity rather than unit heterogeneity.","section":"§4.4, Table 3"},{"comment":"The construction depends on several free parameters whose influence on the results is not analyzed: the size p and selection procedure of the representative sample Y, the intraslice nearest-neighbor count k, the alpha-decay exponent alpha, and the interslice nearest-neighbor count kappa. Because the quantitative comparisons in Table 1 and the correlations in Tables 2 and 3 are central claims, the paper should include a sensitivity analysis or ablation showing that the conclusions are stable over reasonable ranges of these parameters, or at least report p explicitly.","section":"§3.1 and §S6"}],"minor_comments":[{"comment":"The section heading says 'Multiscale PHATE' but the method is called 'Multislice PHATE' elsewhere, including the title and abstract; please use consistent terminology.","section":"§3, heading"},{"comment":"The kernel matrix K is said to be 'an m x nm kernel matrix', but it must be nm x nm since there is one row and column per unit per epoch; please correct this.","section":"§3.2"},{"comment":"In the piecewise definition of K, the term 'K(i)intraslice(tau, upsilon)' should be 'K(i)interslice(tau, upsilon)', and the sentence before the equations refers to 'epsilon' as the 'fixed intraslice bandwidth' when it is later defined as the interslice bandwidth; please fix these typographical errors.","section":"§3.2"},{"comment":"In the supplementary text, the sentence 'the visualizations are qualitatively very similar to those obtained using training data' appears to say 'training data' where 'validation data' is meant, based on the context; please correct this.","section":"§4.4 and §S2"},{"comment":"The column labels in Table 3 appear misaligned in the manuscript text: the row entries number eight, but the header lists nine group labels, and 'Random Pixels' is not clearly represented. Please reformat the table so that each experiment has a distinct, correctly aligned column.","section":"Table 3"},{"comment":"The discussion of Mucha et al. says 'identical fixed weights for all intraslice connections' where 'interslice' is meant, and the conclusion that 'interslice dynamics are not incorporated' is imprecise because that prior work does use fixed interslice weights; please phrase this more carefully.","section":"§2.1"},{"comment":"The PHATE reference cites the 2017 bioRxiv preprint; a published version (Moon et al., Nature Biotechnology, 2019) exists and should be cited if appropriate.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a preprint from 2019 with a clear algorithmic contribution and an open-source implementation, but the quantitative evidence needs substantial strengthening. I would encourage the editor to check the novelty claim ('first method designed explicitly to visualize training dynamics') against related work on activation visualization and trajectory analysis, though this is not the basis for my recommendation. The reported correlation discrepancies and the single-run comparisons are fixable with additional experiments and clearer definitions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: M-PHATE is a real new construction — a multislice kernel with adaptive-bandwidth intraslice affinities and data-dependent interslice affinities, run through PHATE on hidden-unit activations across training epochs. The kernel is clearly specified, the code is promised, and the two vignettes (continual learning, generalization) give visual evidence of things worth seeing: structural collapse at task switches, and reduced heterogeneity when networks memorize. The training-data replication in S2, with ρ=-0.95, is a genuine check that the generalization result does not require validation data.\n\nThe soft spots are all in the quantitative wrapping. Table 1 is one network, one run, no error bars or seeds, and the 'rate of change' metric is never defined. The ARI numbers in Section 4.3 come from an unspecified clustering algorithm. Table 3's ρ=-0.98 is computed over eight conditions and uses summed variance of the PHATE embedding, not direct activation variance; that mixes embedding scale with unit heterogeneity and could be an artifact of the MDS step. The neighborhood-preservation metric is partly circular: it measures how well the embedding preserves the very distances that define the interslice kernel. None of this kills the method; it means the headline numbers are illustrations, not measurements.\n\nI also want to flag the representative sample Y and the per-epoch z-scoring. The paper never says how big Y is or how it is drawn, and z-scoring each unit per epoch strips out activation scale. The authors give a reasonable rationale for the z-score, and the S2 replication helps, but for the generalization correlation this is a real vulnerability.\n\nThe 'first method explicitly designed to visualize NN training dynamics' claim is too strong — time-coupled diffusion maps and related constructions already target evolving data, and the related-work section admits this. The novelty is the specific kernel and the NN application, which is enough.\n\nBottom line: this is a serious paper with a clearly specified method and a promising direction, but the evidence is not yet at the level of the claims. Send it to referees expecting major revisions: multiple seeds, defined metrics, and a direct activation-space control for the variance correlation.","headline":"M-PHATE is a genuinely new multislice kernel construction for visualizing hidden-unit trajectories, but its quantitative headlines rest on single runs and embedding-derived metrics; worth reviewing, not yet established.","tokens_in":13307,"tokens_out":2383,"would_cite":true,"duration_ms":26627,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"M-PHATE makes neural-network training visible by embedding hidden-unit trajectories, revealing catastrophic forgetting and a link between unit heterogeneity and generalization.","keywords":["neural network visualization","multislice kernel","PHATE","dimensionality reduction","catastrophic forgetting","continual learning","generalization","hidden-unit activations"],"falsifier":"Train a fixed three-layer MLP on MNIST under each of the paper's regularization and label-scrambling conditions, then recompute the M-PHATE variance measure while varying the representative sample $Y$ in both size and membership, for instance 100, 500, or 2000 points chosen at random. If the Spearman correlation between summed variance and memorization error does not stay strongly negative across these choices of $Y$, the heterogeneity-generalization link is an artifact of the particular samples shown rather than a property of network training.","tokens_in":12381,"feed_emoji":"🧠","tokens_out":8474,"duration_ms":84534,"temperature":0.7,"pith_summary":"This paper claims that the evolution of a neural network's hidden representations during training can be made visible by building a multislice graph over hidden-unit activations and embedding it with PHATE, a combination the authors call M-PHATE. The aim is to give practitioners a view of learning dynamics that validation loss and accuracy do not provide, and to do so without validation labels. On a feedforward network trained on MNIST, M-PHATE preserves neighborhood structure better than standard t-SNE, Isomap, or diffusion maps and tracks changes in validation loss most faithfully. In continual-learning experiments the visualization shows catastrophic forgetting as a collapse and crossing of hidden-unit trajectories, and in generalization experiments it reveals that higher heterogeneity among hidden units is associated with better generalization, with a Spearman correlation of $\\rho = -0.98$ between summed embedding variance and memorization error.","feed_headline":"M-PHATE maps neural training, exposes forgetting and generalization","feed_subtitle":"It tracks hidden units across training, showing collapse at task switches and diversity when networks generalize.","key_machinery":"The load-bearing construction is the multislice kernel matrix $K$, whose rows and columns index every hidden unit at every epoch. Within an epoch, an adaptive-bandwidth Gaussian kernel (the alpha-decay kernel from PHATE) connects units whose z-scored activation patterns over a sample of inputs are similar; between epochs, a fixed-bandwidth Gaussian kernel connects each unit to itself across time. The kernel is symmetrized, normalized to a random walk, and embedded with PHATE, so the final picture is a diffusion-geometry layout in which time becomes trajectory motion. The paper also uses an Adjusted Rand Index on clusterings of pre- and post-switch slices to quantify structural collapse, and summed per-epoch variance of the embedding as a heterogeneity measure.","core_discovery":"The central discovery is that a time-aware multislice kernel on per-epoch hidden-unit activations, embedded with PHATE, yields a two-dimensional portrait in which each hidden unit is a trajectory over training. The paper argues this portrait exposes mechanisms that scalar metrics hide: in continual learning, the worst-performing optimizers show structural collapse and trajectory crossings at task switches, and the Adjusted Rand Index of cluster assignments before and after a task switch correlates with final validation loss ($\\rho = 0.94$); in generalization, the summed variance of the embedding across epochs correlates with memorization error (Spearman $\\rho = -0.98$, and $\\rho = -0.95$ when only training data are used), so more heterogeneous hidden units coincide with better generalization. Quantitatively, M-PHATE achieves the best intraslice neighborhood preservation among the compared methods and ties t-SNE on interslice preservation, and it has the highest correlation with change in validation loss.","pith_inferences":["Because the method z-scores each unit's activations before building the kernel, it implicitly asserts that the shape, not the magnitude, of activation patterns carries the learning signal; a direct comparison to an unnormalized kernel would test whether scale also matters.","The heterogeneity-generalization link holds across dropout, L1/L2 regularization, random labels, and random pixels, so it may extend to architecture and optimizer comparisons, where it could serve as a validation-free early-stopping or model-selection signal if the correlation proves stable.","The multislice graph construction is not specific to neural networks and could be applied to any dynamical system with repeated measurements over time, such as training dynamics of generative models or drift in biological recordings."],"forward_implications":["M-PHATE can be used during training to inspect learning dynamics from hidden-unit activations alone, so a practitioner does not need validation labels to see when a network is memorizing or losing previously learned structure.","Catastrophic forgetting manifests as structural collapse and trajectory crossings in the embedding; measuring cluster rearrangement with the Adjusted Rand Index gives a proxy for final validation loss ($\\rho = 0.94$).","The summed variance of the M-PHATE embedding across epochs is a heterogeneity score that correlates with memorization error (Spearman $\\rho = -0.98$ on validation-based visualizations, $\\rho = -0.95$ when only training data are used), connecting hidden-unit diversity to generalization.","On the compared feedforward MNIST example, the multislice kernel improves neighborhood preservation over the standard kernel for every dimensionality-reduction method tested, and M-PHATE has the highest correlation with change in validation loss."],"supporting_citations":[{"why":"Defines PHATE, the kernel-based dimensionality-reduction method that M-PHATE applies to the multislice graph.","marker":"[14]"},{"why":"Provides diffusion maps, the diffusion-geometry foundation PHATE builds on.","marker":"[15]"},{"why":"Introduces multislice network community detection by connecting nodes to themselves across slices, the graph idea M-PHATE adapts.","marker":"[26]"},{"why":"Shows how to align diffusion maps over changing data, a precursor the paper contrasts with its interslice-aware kernel.","marker":"[20]"},{"why":"Supplies the continual-learning scenarios and baseline training schemes used in the catastrophic-forgetting experiments.","marker":"[29]"},{"why":"Establishes the learning-versus-memorization distinction that motivates the generalization experiments.","marker":"[34]"},{"why":"Defines dropout, whose effect on hidden-unit heterogeneity M-PHATE is used to reinterpret.","marker":"[35]"},{"why":"Provides t-SNE, one of the standard visualization baselines compared against M-PHATE.","marker":"[12]"},{"why":"Provides Isomap, the other standard visualization baseline in the neighborhood-preservation comparison.","marker":"[13]"},{"why":"Supplies multidimensional scaling, the embedding step PHATE uses to produce the final low-dimensional layout.","marker":"[11]"}],"fun_headline_variants":["M-PHATE maps training dynamics and exposes forgetting","New visualization shows why networks forget and generalize","M-PHATE tracks hidden units across training epochs","Visualizing neural training reveals catastrophic forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the chosen sample of input points, together with per-epoch z-scoring of each unit's activations, preserves the signal that separates learning from memorization; if that sample or normalization discards the relevant information, the visual patterns and correlations will not generalize.","fun_headline_variants_meta":{"raw":{"variants":["M-PHATE maps training dynamics and exposes forgetting","New visualization shows why networks forget and generalize","M-PHATE tracks hidden units across training epochs","Visualizing neural training reveals catastrophic forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000307,"raw_usage":{"total_tokens":1763,"prompt_tokens":954,"completion_tokens":809,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":760}},"tokens_in":570,"tokens_out":809,"duration_ms":8858,"temperature":1.0,"reasoning_tokens":760,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:32:58.013371+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a fixed three-layer MLP on MNIST under each of the paper's regularization and label-scrambling conditions, then recompute the M-PHATE variance measure while varying the representative sample $Y$ in both size and membership, for instance 100, 500, or 2000 points chosen at random. If the Spearman correlation between summed variance and memorization error does not stay strongly negative across these choices of $Y$, the heterogeneity-generalization link is an artifact of the particular samples shown rather than a property of network training.","supporting_citations":[{"cited_title":"Visualizing transitions and structure for high dimensional data exploration","cited_arxiv_id":null,"evidence_quote":"Defines PHATE, the kernel-based dimensionality-reduction method that M-PHATE applies to the multislice graph."},{"cited_title":"Community structure in time-dependent, multiscale, and multiplex networks","cited_arxiv_id":null,"evidence_quote":"Introduces multislice network community detection by connecting nodes to themselves across slices, the graph idea M-PHATE adapts."},{"cited_title":"Diffusion maps for changing data","cited_arxiv_id":null,"evidence_quote":"Shows how to align diffusion maps over changing data, a precursor the paper contrasts with its interslice-aware kernel."},{"cited_title":"Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S","cited_arxiv_id":null,"evidence_quote":"Establishes the learning-versus-memorization distinction that motivates the generalization experiments."},{"cited_title":"Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov","cited_arxiv_id":null,"evidence_quote":"Defines dropout, whose effect on hidden-unit heterogeneity M-PHATE is used to reinterpret."},{"cited_title":"A global geometric framework for nonlinear dimensionality reduction","cited_arxiv_id":null,"evidence_quote":"Provides Isomap, the other standard visualization baseline in the neighborhood-preservation comparison."},{"cited_title":"Multidimensional scaling","cited_arxiv_id":null,"evidence_quote":"Supplies multidimensional scaling, the embedding step PHATE uses to produce the final low-dimensional layout."}],"review_version":1}