{"id":"d0a091be-d3b2-467a-adc9-03f3504c6ae9","arxiv_id":"2607.10285","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Autoencoders trained on Ising spin configurations learn large-scale magnetization before small-scale energy features; deep models often arrest before the energy stage, and recursive self-application reveals stable latent topology for learned concepts.","lead":"This paper trains autoencoders on Ising-model spin patterns and watches, step by step, which physical features the networks learn first. It finds that models reproduce overall magnetization before finer energy structure, that deep networks often get stuck before the energy stage, and it uses a self-recursion trick to watch concepts form inside the latent space.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fig. 23 directly contradicts the central claim: energy losses at no scale consistently decrease over training, yet Sec. V asserts an energy regime with monotone decrease across all scales.","rationale":"The reader's weakest assumption focuses on validation distribution shift between training temperatures and the full 15-temperature validation set. That is a plausible concern, but I find a more direct, load-bearing problem: the energy-loss evidence in the paper's own Fig. 23 contradicts the central claim of a monotone-decreasing energy regime. The reader mentions this only as rationale point (1) and does not make it the central attack. The two concerns are related, since both question whether the claimed sequence of regimes is real, but the Fig. 23 contradiction is internal to the paper and does not depend on interpreting distribution shift. A concrete re-analysis of the energy-loss time series can settle whether any contiguous monotone-decreasing interval exists at all scales, and if so whether it is robust per temperature. The conditional verdict stands: the paper has valuable empirical observations and released code/data, but the central claim as stated is not acceptable without reconciling Sec. V with Fig. 23.","tokens_in":33785,"tokens_out":7708,"duration_ms":82844,"concrete_test":"Using the released DaRUS checkpoints, recompute the energy-loss curves for the MB8DXWX ensemble (b=8, λ=10^-4, all depths/widths) at each coarse-graining kernel. For each run, estimate the sliding-window time derivative of L_energy(k, t). Define an energy regime as a contiguous training interval where the median slope is negative at every scale k. Check whether such an interval exists for d=4 and whether it aligns with the large-scale magnetization minimum around step 10^4. If no such interval exists, Sec. V's monotone-decrease claim fails; if it exists only in the temperature-averaged validation set, repeat per temperature to distinguish a genuine regime from distribution shift.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Abstract and Sec. V) requires that a distinct energy-learning regime be visible in scale-resolved loss curves. The paper's own appendix contradicts this. In Sec. VI E 3, describing Fig. 23 — the energy analogue of Fig. 6 for the same MB8DXWX ensemble used in the main narrative — the text states: \"not a single loss on any of the scales in Fig. 23 consistently decreases over training. Notably, the losses even increase compared to the randomly initialized model in most cases.\" Likewise, the caption says the final average energy loss is larger than the initial loss for most models, except at intermediate depth (d=4). Sec. V, however, asserts that \"the scale-dependent loss evolution revealed two distinct dynamical regimes in which either the magnetization or the energy losses decrease monotonically across all scales.\" If \"decrease across all scales\" means \"decrease over training time at every kernel size,\" Fig. 23 falsifies the energy regime as stated. If it instead means \"decrease as kernel size increases,\" it is a static ordering that does not establish a temporally sequential energy regime. Either reading leaves the strongest concrete assertion unsupported as written. This is an internal inconsistency, not merely a distribution-shift concern, and it should be resolved before the central claim is accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how deterministic ReLU autoencoders trained on 2D Ising spin configurations learn macroscopic concepts, using spatially coarse-grained validation losses for magnetization and energy. Based on 972 training runs, it claims that learning proceeds through two sequential dynamical regimes — first magnetization, then energy — and that depth, width, and learning rate control the transition, with deep models trained at moderate/fast rates becoming arrested before the energy regime. The paper also introduces an analysis of self-recursive latent dynamics and Spearman-rank 'topological ordering' to connect reconstruction errors to internal concept formation.","tokens_in":34085,"tokens_out":4140,"duration_ms":50159,"significance":"If the central claim were established, the paper would provide a substantial empirical contribution: a hyperparameter-controlled, scale-resolved picture of concept formation in autoencoders on a physically grounded dataset. Strengths include the large ablation study, publicly available code/checkpoints, comparisons to trivial baselines (dataset mean and sample mean), and the fact that the two regimes are measured rather than fitted. However, the central energy-regime claim is directly contradicted by the paper's own Fig. 23 and Appendix §VI E 3, which state that no energy loss at any scale consistently decreases over training. This internal inconsistency prevents acceptance of the paper in its current form; the remaining distribution-shift confound also weakens the causal interpretation of the magnetization-to-energy trade-off.","major_comments":[{"comment":"The central claim is internally inconsistent. The Abstract states that there is a regime 'in which energy is learned across scales,' and §V asserts that 'the magnetization or the energy losses decrease monotonically across all scales.' But §VI E 3 says, describing Fig. 23 for the same MB8DXWX ensemble used in the main narrative, 'not a single loss on any of the scales in Fig. 23 consistently decreases over training. Notably, the losses even increase compared to the randomly initialized model in most cases.' The caption adds that final average energy loss is larger than initial for most models, except at d=4. If 'decrease across all scales' means temporal decrease at each kernel size, Fig. 23 falsifies the energy regime as stated; if it means the static ordering of loss versus kernel size, that is not a sequential dynamical regime. Either way, the strongest concrete assertion in §V is uns","section":"Abstract; §V; §VI E 3 and Fig. 23"},{"comment":"The magnetization-to-energy sequence and the bottleneck-causes-trade-off interpretation are confounded by a validation distribution shift. Training uses only the five temperatures near T_c, while validation losses are averaged over all 15 temperatures. The paper itself notes in §IV B 3 that the highest-energy samples 'are only present in the validation data.' The rise of the large-scale loss after its minimum in Fig. 6 could therefore be an artifact of the model degrading on far-from-training temperatures, rather than a bottleneck-induced representational trade-off. Per-temperature validation curves are not used in the main narrative. Without ruling out distribution shift, the claim in §V that 'the bottleneck constraint is the cause of the trade-off' is not established. The authors should either show per-temperature versions of the key scale-resolved curves or explicitly qualify the trad","section":"§III A 2; §IV B 2; §IV B 3; §V"},{"comment":"The concluding sentence of §IV C 2 states that stability under recursion 'is a prerequisite for consistently improving their reconstruction.' This causal claim is stronger than what the Spearman-rank analysis supports: the analysis shows a temporal concurrence between high stable rank correlation and decreasing magnetization loss, but correlation at the same training checkpoints does not establish precedence or causation. This is not the primary claim of the paper, but the wording should be softened to 'associated with' or 'precedes in the observed runs' unless a temporal ordering is demonstrated.","section":"§IV C 2 and Fig. 10"}],"minor_comments":[{"comment":"Grammar/punctuation issues: 'The first exhibits error fluctuations ordered to scale and learns global averages only; The second' uses a semicolon followed by a capital 'The,' and the sentence is run-on. Recommend copyediting.","section":"Abstract"},{"comment":"The notation for coarse-grained magnetization is used as M(k) but the definition of the kernel set K_k^i is not fully specified in the main text; please clarify the relation between kernel size k and the 'a×a' notation in L_{M(k)} = L_{k=a×a}(...).","section":"§IV B 2"},{"comment":"The caption refers to 'w=1025 models'; the main text and Table I only define widths 256, 512, and 1024. This is likely a typo and should be corrected.","section":"Fig. 20 caption"},{"comment":"The appendix's observation that only d=4 models show a final average energy loss lower than the initial loss is important; it should be discussed in the main text, not buried in a caption, because it bears directly on the energy-regime claim.","section":"§VI E 3 / Fig. 23"}],"recommendation":"major_revision","confidential_remarks":"The paper is thorough and the experimental apparatus is strong, but the central claim about an energy-learning regime is contradicted by the paper's own Fig. 23 and Appendix text. This is not a matter of presentation; the conclusion as stated in the Abstract and §V would need to be substantially revised, or the analysis of energy losses redone, before the paper can be accepted. The distribution-shift confound adds a second layer of required work. I recommend major revision rather than rejection because the magnetization regime, the arrest phenomenon, and the observational data appear sound; the authors could plausibly reframe the paper around what the energy-loss data actually show."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know two things about arXiv:2607.10285. First, it is a substantial empirical study: 972 runs, publicly released code and data, and a genuinely new way of looking at what autoencoders learn—decomposing reconstruction losses by coarse-graining scale and probing latent space via self-recursive flow. Second, its central claim, as written in the abstract and Sec. V, is not supported by the paper's own appendix. That's not a minor quibble.\n\nWhat's good. The magnetization-first result is credible. Coarse-scale averages are learned before fine-scale structure, large-scale losses hit a minimum and then rise as smaller scales improve. The deep-model arrest—networks stuck predicting the dataset average—is documented carefully, with comparisons to trivial baselines and run variance. The self-recursion/Spearman analysis is a real methodological addition, and the authors are honest about energy not being represented as a single latent dimension. The empirical contribution is solid.\n\nThe soft spot is load-bearing. Sec. V says the scale-dependent loss evolution revealed two distinct regimes in which magnetization or energy losses decrease monotonically across all scales. But in Sec. VI E 3, describing Fig. 23—the energy analogue of Fig. 6 for the same ensemble—the authors write: 'not a single loss on any of the scales in Fig. 23 consistently decreases over training.' The final average energy loss is larger than the initial loss for most models. So either 'decrease across all scales' means 'with kernel size' (a static ordering, not a temporally sequential regime), or it is false over training time. Either way, the abstract overstates what the data show.\n\nThere's a second concern, related: training was on five critical-temperature datasets, while validation covered fifteen temperatures. The paper itself notes high-energy samples 'are only present in the validation data.' The large-scale loss rise that defines the magnetization-to-energy boundary could be distribution shift rather than a representational trade-off. Per-temperature validation curves would settle this; they're not in the main narrative.\n\nAlso, the recursion premise about convergence in a 'finite space' is wrong for a continuous latent space. Minor, but needs fixing.\n\nWho should read this: people working on learning dynamics, interpretable ML, and physics-informed ML. The diagnostic toolkit is worth engaging, and the magnetization/arrest results are useful. But the headline claim needs to be rewritten to match Fig. 23, with per-temperature analysis added.\n\nRecommendation: send to peer review. This deserves referee time, not a desk reject, but the referees should insist on reconciling the abstract with Fig. 23 before acceptance.","headline":"A substantial empirical study of scale-resolved autoencoder dynamics whose headline 'energy regime' is contradicted by its own Fig. 23; the magnetization result is solid, but the central claim needs rework.","tokens_in":34541,"tokens_out":3183,"would_cite":false,"duration_ms":34869,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that autoencoders trained on Ising-model spin configurations learn macroscopic concepts in a fixed order — first magnetization, then energy — and that this order can be read off from reconstruction losses decomposed across","keywords":["autoencoders","Ising model","learning dynamics","coarse-graining","magnetization","energy","representation learning","critical phenomena"],"falsifier":"Evaluate the coarse-grained validation losses separately for each of the fifteen temperatures. If the global minimum in the largest-scale loss, and the subsequent rise, appears only at temperatures far from the training set (T ≥ 2.5) and not at the five training temperatures, the claimed magnetization-then-energy sequence and bottleneck-induced trade-off would not hold; equivalently, train and validate on the same critical temperatures and check whether the trade-off persists.","tokens_in":33623,"feed_emoji":"🧲","tokens_out":6388,"duration_ms":62051,"temperature":0.7,"pith_summary":"This paper tries to show that an autoencoder — a neural network that compresses input through a narrow bottleneck and reconstructs it — trained on spin configurations of the two-dimensional Ising model (a grid of interacting spins) does not learn all features at once: its reconstruction errors, decomposed across spatial coarse-graining scales, fall into two sequential dynamical regimes. In the first regime, only global averages — the magnetization — are learned across all scales; in the second, the model gradually resolves the small-scale fluctuations needed to represent the energy. The transition between regimes is governed by model depth, width, bottleneck size, and learning rate, and deeper models trained at moderate or fast rates become 'arrested' before ever reaching the energy regime. If correct, this gives a concrete, physically grounded picture of how unsupervised neural networks build macroscopic concepts from microscopic data, and a scale-based diagnostic for when learning is succeeding or stalled.","feed_headline":"Autoencoders learn magnetization before energy, scale by scale","feed_subtitle":"Scale-resolved losses reveal two learning regimes — and a stall mode in deep networks.","key_machinery":"The load-bearing tool is a scale-resolved loss: reconstruction error is computed not just on raw pixels but on averages over square neighborhoods (kernels) of growing size, turning the loss into a function of the coarse-graining scale. A second probe is self-recursion: the trained network is applied to its own output to generate latent-space trajectories; a rank correlation between latent distances and an observable's difference measures whether the latent space is topologically ordered by that concept. Together these let the authors watch the order in which macroscopic concepts form.","core_discovery":"The central claim is that, when the mean-squared reconstruction error is averaged over square patches of growing size on the 16x16 lattice, successful training shows two distinct phases: a magnetization phase in which losses at all scales decrease monotonically and the output is a spatially uniform version of the input, followed by an energy phase in which small-scale structure is resolved and losses become scale-dependent. The transition is marked by a minimum in large-scale losses, which then rise as small-scale features improve — a representational trade-off that the authors attribute to the information bottleneck. Deep autoencoders (depth 16) trained at learning rates of 1e-4 or 1e-3 nev","pith_inferences":["If the magnetization-then-energy order reflects the data's correlation structure rather than anything specific to Ising systems, the same scale-resolved loss probe could be used on images, turbulence data, or other multiscale datasets to detect when a model is about to resolve finer scales.","The recursion-dynamics measurement could double as a practical monitoring heuristic: a latent space whose topology for a target concept is stable under recursion may be a better early-stopping signal than the raw validation loss.","The per-temperature validation analysis lives only in the supplementary material; using it in the main narrative would clarify whether the trade-off is intrinsic or a distribution-shift artifact. This is my inference, not the paper's claim.","One could test whether an explicitly curriculum-driven training schedule — feeding low-temperature samples first, then progressively warmer ones — reproduces or accelerates the magnetization-to-energy sequence, which would suggest the emergent order is an implicit curriculum."],"forward_implications":["For shallow models (depths 1 and 4), learning global spin averages before local fluctuations appears to be a robust ordering, so simpler statistics of the data are mastered before finer correlations.","The large-scale loss minimum and subsequent rise constitutes a scale-resolved signature of a representational trade-off, and its strength decreases as the bottleneck grows.","Deep models at intermediate and fast learning rates exhibit an arrested regime in which they output only the dataset average; this regime is identifiable from flat, scale-invariant run variance.","The robustness of the latent-space topological ordering under recursion can serve as an indicator that a concept has actually been learned, not merely that the reconstruction loss dropped.","The joint output distribution of magnetization and energy shows that high-energy validation configurations are systematically underestimated, indicating that the bottleneck limits resolution of small-scale features even in successful models."],"fun_headline_variants":["Autoencoders learn magnetization, then energy, in stages","Two-stage learning: Ising autoencoders master magnetization before energy","Deep autoencoders stall before learning energy in Ising model","Scale-resolved losses show magnetization first, then energy in autoencoders","Ising autoencoders: magnetization phase, then energy phase, or stall"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The magnetization-to-energy sequence is read off from validation losses averaged over all fifteen temperatures, even though training used only the five critical-adjacent temperatures, so the apparent trade-off could be an artifact of distribution shift rather than an intrinsic bottleneck effect.","fun_headline_variants_meta":{"raw":{"variants":["Autoencoders learn magnetization, then energy, in stages","Two-stage learning: Ising autoencoders master magnetization before energy","Deep autoencoders stall before learning energy in Ising model","Scale-resolved losses show magnetization first, then energy in autoencoders","Ising autoencoders: magnetization phase, then energy phase, or stall"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00111,"raw_usage":{"total_tokens":4440,"prompt_tokens":698,"completion_tokens":3742,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":3651}},"tokens_in":442,"tokens_out":3742,"duration_ms":26752,"temperature":1.0,"reasoning_tokens":3651,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:18:59.745517+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the coarse-grained validation losses separately for each of the fifteen temperatures. If the global minimum in the largest-scale loss, and the subsequent rise, appears only at temperatures far from the training set (T ≥ 2.5) and not at the five training temperatures, the claimed magnetization-then-energy sequence and bottleneck-induced trade-off would not hold; equivalently, train and validate on the same critical temperatures and check whether the trade-off persists.","supporting_citations":[],"review_version":2}