{"id":"d70db25d-1794-49bf-840c-ed92c5c416c7","arxiv_id":"2511.05350","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Noise-augmented training of an audio autoencoder organizes representations so that perceptually salient information survives in coarse structures, improving musical surprisal estimates and EEG prediction.","lead":"This paper adds noise to the compressed representations (latents) of an audio autoencoder during training, and claims this makes the most perceptually important musical information live in the coarsest parts of the representation. The authors show this 'perceptual hierarchy' improves two tasks: estimating how surprising a note is, and predicting brain (EEG) responses to songs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Latent spectral SNR assumption unverified: if Music2Latent latents aren't power-law, noise doesn't remove fine structures first, undercutting the perceptual-hierarchy interpretation.","rationale":"The paper is internally consistent and includes strong empirical controls (NT=D, multiple variants, external benchmarks). The most vulnerable point is the unverified transfer of spectral SNR properties from natural signals to learned audio latents. This is an empirical precondition, not a logical inconsistency or a matter of consensus. If the latent spectrum is not power-law, the claimed perceptual hierarchy is unsupported, and the results reduce to demonstrating that noise-augmented training improves robustness—still interesting but not the central claim. The proposed test is feasible with the released pretrained weights and directly targets the mechanism. The reader's weakest_assumption identified the same issue, so I agree. No change to the CONDITIONAL verdict is needed; the paper should remain conditional pending this verification.","tokens_in":10285,"tokens_out":6542,"duration_ms":57662,"concrete_test":"Encode the Section 4 MusicCaps clips with the aligned (LayerNorm m=-1) and unaligned (NT=∅ and NT=D) encoders. Compute the temporal power spectral density (PSD) of the latent sequences (per dimension or first principal components). If the PSD decays as a power law with frequency, the spectral SNR premise holds; if it is flat (exponent ≈ 0), noise removes all frequencies equally, so low SNR cannot be equated with coarse structures. Additionally, compute per-frequency SNR after adding Gaussian noise at SNRs {∞,4,1,0.25} and confirm that high frequencies lose SNR faster than low frequencies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that noised-latent training creates a perceptual hierarchy with salient information in coarse structures—rests on the spectral SNR property of diffusion processes: adding Gaussian noise destroys fine (low spectral power) structures before coarse (high spectral power) ones, because natural signals have a decaying power spectrum (refs. 14, 15, 42). The paper assumes this transfers to Music2Latent latents, but never verifies that the learned latent representation actually has a power-law decaying temporal power spectrum. Section 3 states this as the mechanism for hierarchical alignment; Section 4 then interprets SNR-dependent reconstruction quality and IDyOM/EEG results as evidence of hierarchy. If the latent z is spectrally flat (e.g., due to LayerNorm or the CAE training), white Gaussian noise does not preferentially remove fine structures. The observed improvements at intermediate noise levels could then be explained by the decoder's learned denoising robustness (even with the NT=D control, which only isolates decoder adaptation) or by the diffusion model's training, rather than by a coarse-to-fine perceptual organization in the encoder. This is the load-bearing assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes that training an audio autoencoder to reconstruct clean inputs from noised versions of its latents, together with perceptually motivated losses, induces a perceptual hierarchy in which the most perceptually salient information is captured in coarse latent structures and less salient information in finer structures. The authors finetune Music2Latent with such noise-augmented latents (Eq. 1), fixing latent variance via LayerNorm, and evaluate the resulting representations through three lines of evidence: reconstruction quality at different signal-to-noise ratios (Section 4), correlation of diffusion-model information content with IDyOM pitch surprisal (Section 5), and neural encoding of EEG responses to sung music (Section 5). They report consistent improvements over an unaligned baseline and over the prior method [13], especially at intermediate noise levels, and release pretrained weights.","tokens_in":10642,"tokens_out":2185,"duration_ms":23325,"significance":"If the central claim holds, the paper makes a useful contribution by connecting latent diffusion noise schedules to perceptual organization, with practical implications for audio representation learning and music surprisal estimation. The strengths include the use of two external, perceptually validated benchmarks (IDyOM, EEG), a decoder-only control condition (NT=D) that addresses part of the train/inference mismatch, and the public release of pretrained weights. The reported improvements are nontrivial and the results are internally consistent across reconstruction, behavioral correlation, and neural encoding. However, the main interpretation rests on an assumption about the spectral properties of the learned latents that is not directly verified, and the model-selection procedure is partly based on the same evaluation tasks.","major_comments":[{"comment":"The paper's central mechanism is the spectral SNR property of diffusion noise: adding Gaussian noise destroys fine structures (low spectral power) before coarse structures (high spectral power), as established for natural signals with power-law spectra (refs. [14,15,42]). The authors assume this transfers to Music2Latent latents, but they never verify that the learned latent representation z actually has a power-law decaying temporal power spectrum. If z is spectrally flat—possible due to LayerNorm, TanH, or CAE training—then Gaussian noise does not preferentially remove fine structures, and the observed improvements at intermediate noise levels could instead reflect decoder robustness or increased redundancy, not a coarse-to-fine perceptual hierarchy. This is load-bearing for the paper's main claim. I request a direct test: plot the average power spectrum of z=E(x) for held-out audio, o","section":"Sections 2–4, Eq. (1)"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has a real empirical result — noise-augmented autoencoder training with perceptual losses seems to put information relevant to musical surprisal and EEG prediction into more noise-robust parts of a learned latent space — but the mechanistic claim about a 'perceptual hierarchy' is inferred, not directly shown, and one load-bearing assumption (that diffusion noise removes fine structures before coarse ones in learned latents) is never tested.\n\nWhat's actually new: combining [16]'s noise-augmented training with perceptual losses for music audio, and testing it on two externally grounded tasks (IDyOM pitch surprisal, EEG cortical tracking). The decoder-only control (NT=D) is a good check that the gains aren't just the decoder learning to denoise. The appendix is unusually transparent about model selection and shows all variants, not just the chosen one.\n\nSoft spots: the paper relies on spectral SNR properties established for natural signals and assumes they transfer to Music2Latent latents. If the latent is spectrally flat — and LayerNorm might push it that way — white noise does not preferentially remove fine structures. The authors explicitly avoid measuring hierarchy in the latent space. That said, the result doesn't fully depend on that specific mechanism: the encoder could allocate variance to perceptually important dimensions, which is itself a hierarchy. So the interpretation is underdetermined rather than wrong. Second, the main variant (LayerNorm, m=-1) was selected after looking at downstream benchmark performance; that is a form of evaluation fitting, though the paper reports all variants so readers can judge for themselves. Third, training code, the private singing-voice finetune set, and the EEG data are not released, so full reproduction is not possible.\n\nBottom line: a solid workshop paper with an interesting, partially supported claim. The right next step is to test the latent spectral assumption directly — measure the power spectrum of z, verify noise destroys low-power components first, and probe which latent dimensions carry pitch vs timbre. If that holds, this would be a genuinely useful contribution to latent diffusion for audio. I would send it to a workshop or short-paper peer review; for a full conference I would want the mechanism verified or the claim softened.","headline":"Real empirical gains, but the perceptual-hierarchy mechanism is inferred rather than shown — the latent spectral assumption needs a direct test.","tokens_in":11078,"tokens_out":3695,"would_cite":true,"duration_ms":30330,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Noise-augmented training of audio autoencoders produces latent spaces ordered by perceptual importance, so pitch and other salient information survives in coarse structures and improves surprisal and EEG prediction.","keywords":["noise-augmented autoencoders","perceptual hierarchy","latent diffusion","musical surprisal","pitch expectancy","EEG neural encoding","audio representation learning","consistency autoencoder"],"falsifier":"Corrupt only the coarsest latent components versus only the finest components at equal total power and measure reconstruction quality: the hierarchy claim predicts a large drop in ViSQOL and SI-SDR when coarse components are corrupted and a small drop when fine components are corrupted. If both corruptions hurt about equally, the observed gains come from decoder robustness to noise, not from perceptual structure in the encoder. A second check: use a diffusion noise process that removes low frequencies before high frequencies; the spectral-SNR assumption predicts this ordering should destroy th","tokens_in":10211,"feed_emoji":"🎵","tokens_out":6326,"duration_ms":55019,"temperature":0.7,"pith_summary":"This paper argues that an audio autoencoder can be trained so its latent code is organized by perceptual importance: the most salient information is stored in the coarsest components, and progressively finer components carry less perceptually relevant detail. The training trick is to ask the autoencoder to reconstruct clean audio from noised versions of its own latent codes, while keeping the latent variance fixed so the noise level is meaningful. The authors show this alignment by decoding latents corrupted to different signal-to-noise ratios: the aligned code preserves perceptual quality better than an unaligned one at every noise level. They then show the benefit for latent diffusion: information-content estimates computed from the aligned space correlate more strongly with a perceptually validated pitch-expectancy model and with EEG responses to sung music, peaking at intermediate noise levels. If right, this offers a general way to impose perceptual structure on learned audio representations without changing the task.","feed_headline":"Noise training aligns audio latents with how we hear music","feed_subtitle":"Coarse codes carry the salient sound, improving pitch-surprisal estimates and EEG prediction.","key_machinery":"The central mechanism is noise-augmented reconstruction training: during autoencoder training, latent vectors are interpolated with Gaussian noise at random strengths and the decoder must reconstruct the clean input, which forces the encoder to place information according to how easily it can be recovered from noise. Two details make the mechanism work: a perceptual reconstruction loss defines what 'salient' means, and a layer-normalization bottleneck fixes the latent variance so the expected signal-to-noise ratio stays constant, preventing the encoder from inflating its outputs to dodge the noise. The paper also relies on the spectral signal-to-noise ratio property of diffusion noise—noise","core_discovery":"After finetuning an audio consistency autoencoder to reconstruct clean input from noised latent vectors (z'=(1−t)z + t·noise, at strengths spanning the latent-diffusion range), the encoding is claimed to become hierarchically aligned with perception: 'coarse' means 'survives heavy noise,' because diffusion noise erases low-power structures first. Reconstructions from latents corrupted to SNR 4, 1, and 0.25 retain higher perceptual quality (ViSQOL, SI-SDR) than unaligned latents, and at the lowest SNR the aligned decoder produces more plausible but invented content. An autoregressive rectified-flow model in the aligned space yields pitch-surprisal information content correlating better with a","pith_inferences":["If the perceptual loss is changed, the induced hierarchy should re-rank accordingly; for example, a loss emphasizing timbre should move timbre into coarser structures, making the method a potential tool for controllable perceptual organization of latent spaces.","The variance-fixing bottleneck is likely load-bearing: without it, the encoder can grow latent variance to neutralize the noise, so the perceptual hierarchy may not emerge. Replications should monitor latent SNR during training.","The low-SNR behavior of the aligned decoder—plausible but invented content—suggests the alignment doubles as a generative prior; the same encodings may improve generation, inpainting, or music continuation, not just surprisal measurement.","A direct test of the hierarchy would corrupt only coarse versus only fine latent components; the paper's account predicts strongly asymmetric effects on perceptual metrics."],"forward_implications":["Perceptual quality of reconstructions from heavily noised latents is higher when the autoencoder was trained with noise augmentation than under conventional, unaligned training.","Pitch-surprisal estimates computed in the aligned latent space correlate more strongly with a perceptually validated symbolic pitch model than unaligned latents or the baseline, with the correlation peaking at intermediate noise levels—evidence that pitch information sits in coarse structures.","Information content from aligned latents explains significantly more unique variance in EEG responses to sung music than unaligned latents, with widespread fronto-central effects.","Because diffusion models denoise coarse structures most accurately, aligning salient perceptual information with coarse structure should improve latent diffusion decoding generally, not only for surprisal estimation.","The intermediate-noise peak identifies a useful operating point: enough noise to strip less relevant detail, not so much that pitch information is lost."],"fun_headline_variants":["Noise-trained audio codes mirror how we hear music","Audio autoencoders get a perceptual hierarchy from noise","Heavy noise reveals perceptually salient sound in music codes","Noise-augmented training aligns audio latents with hearing","Perceptual music representations via noise-based autoencoder"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The premise that carries the paper is that noise level tracks structure coarseness in the learned latent space: adding stronger noise removes progressively finer latent structures, so what survives the most noise is exactly the coarse perceptual code.","fun_headline_variants_meta":{"raw":{"variants":["Noise-trained audio codes mirror how we hear music","Audio autoencoders get a perceptual hierarchy from noise","Heavy noise reveals perceptually salient sound in music codes","Noise-augmented training aligns audio latents with hearing","Perceptual music representations via noise-based autoencoder"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000137,"raw_usage":{"total_tokens":942,"prompt_tokens":653,"completion_tokens":289,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":397,"completion_tokens_details":{"reasoning_tokens":211}},"tokens_in":397,"tokens_out":289,"duration_ms":4530,"temperature":1.0,"reasoning_tokens":211,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T23:28:51.254313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Corrupt only the coarsest latent components versus only the finest components at equal total power and measure reconstruction quality: the hierarchy claim predicts a large drop in ViSQOL and SI-SDR when coarse components are corrupted and a small drop when fine components are corrupted. If both corruptions hurt about equally, the observed gains come from decoder robustness to noise, not from perceptual structure in the encoder. A second check: use a diffusion noise process that removes low frequencies before high frequencies; the spectral-SNR assumption predicts this ordering should destroy th","supporting_citations":[],"review_version":1}