{"id":"dc4c5627-e8eb-431b-9417-b8ba694a0bf0","arxiv_id":"2505.12235","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"NOFT fine-tunes the Gaussian noise latent at test time with a 14K-parameter optimal-transport information bottleneck, producing diverse but content-preserving variations from a single diffusion model without extra control signals.","lead":"A new test-time module, NOFT, modifies the random noise seed of a diffusion model to create image and 3D asset variations that stay close to the original while exploring new structures and textures. It uses a small optimal-transport and information-bottleneck network, trained in about 10 minutes with only 14K parameters, instead of fine-tuning the model or using external control maps.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The objective never rewards transfer from NDiv: Eq. 10 drops the I(Y;Z) term, and Eq. 11's per-location mixing makes the output statistically just another Gaussian noise, so the claimed topology transfer is unsupported.","rationale":"The reader's weakest assumption is right, and the paper's own equations make the problem sharper than the reader stated. The IB objective in Eq. 4 is explicitly max I(Y;Z) − βI(X;Z), but the implementation in Eq. 10 minimizes only the compression term; no I(Y;Z) is ever defined or optimized. Lnoise (Eq. 15) supervises reconstruction of NOrig, so the network is trained to stay close to the source noise, not to inherit structure from NDiv. The diversity source NDiv appears only in Eq. 11 as an independent Gaussian; mixing two independent white Gaussian noise fields per-location produces another white Gaussian noise field (for scalar or elementwise λ), so the output cannot carry NDiv's image topology unless some cross-noise alignment mechanism is introduced. None is specified: FSA is applied to NOrig only in Eq. 9. The qualitative results may still be useful as stochastic variation, but they cannot be taken as evidence of topology transfer without the missing baseline and an explicit objective term. I agree with the reader's CONDITIONAL verdict; the conditions should include the direct interpolation baseline and a permutation test for NDiv. This is not an attack on the authors' honesty; it is a request for the one experiment that distinguishes learning from interpolation.","tokens_in":12128,"tokens_out":9087,"duration_ms":98689,"concrete_test":"Using the same prompts, seeds, and evaluation protocol as Table 1 and Figure 5, replace NOFT with direct per-location interpolation N_t = (1−t)NOrig + tNDiv, sweeping t = 0.05, 0.1, 0.2, 0.4, 0.6, 0.8, and also optimize a scalar/channel-wise t with the same Lnoise objective. Compare self-sim, DINO-I, L1, and user fidelity/diversity on matched samples. If the best interpolation matches NOFT within sampling noise, then Eq. 11's mixing—not the OTIB training or Sinkhorn alignment—is doing the work, and the central topology-transfer claim is unsupported. A secondary check within the same protocol: shuffle NDiv's spatial layout before mixing; since Gaussian noise is spatially exchangeable, the output distribution should be unchanged if the method is merely adding noise, whereas the paper's claim predicts dependence on NDiv's specific spatial pattern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that NOFT transfers the local topological statistics of NDiv's image into NOrig. The objective contains no term that rewards this. Eq. 4 defines the IB objective as max I(Y;Z)−βI(X;Z), but Sec. 4.2.1 replaces it with Eq. 10, min βI(R;Z), deleting the task-relevant term I(Y;Z). The only supervision is Eq. 15, Lnoise = ||NNOFT − XOrig||² (XOrig/NOrig typo), which penalizes deviation from the source noise. There is no term that measures or maximizes information from NDiv, so nothing in the loss teaches the network to transfer NDiv's structure. NDiv enters only as the independent Gaussian ϵ in Eq. 11, mixed per-location via Z = λR + (1−λ)ϵ. If λ is a scalar or an elementwise learned mask, the output is still a white Gaussian noise field: per-location interpolation of two i.i.d. Gaussian noises is statistically just another Gaussian sample. The Sinkhorn Attention FSA operates on NOrig alone (Eq. 9), so no cross-noise alignment between NOrig and NDiv appears anywhere. The paper's own ablation without SA shows local degradation, but that only shows FSA reshapes the source noise; it does not establish that NDiv's spatial pattern is transferred. The observed diversity is therefore equally explainable as ordinary stochastic variation around NOrig, and the claimed 'local topological statistic from I2' transfer has no mechanism in the objective.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NOFT, a plug-and-play module that fine-tunes the initial noise latent of a frozen diffusion model. The module combines a Sinkhorn Attention (FSA) and an information bottleneck (IB) to mix a source noise NOrig with a diversity noise NDiv, aiming to preserve the structure and appearance of the source image while injecting diversity. The method is evaluated on 2D and 3D asset generation, compared against several baseline control methods, and claims to transfer the local topological statistics of the second noise's image into the generated image. The paper reports strong qualitative results, quantitative metrics in Table 1, and a small user study.","tokens_in":12417,"tokens_out":5920,"duration_ms":61216,"significance":"If the claims were fully supported, NOFT would be a significant lightweight contribution: a 14K-parameter, 10-minute test-time module that controls content-diversity trade-offs without external control signals or model fine-tuning. The paper has strengths: it addresses an interesting and timely problem, provides extensive qualitative comparisons across multiple architectures, and the modular design is clean. However, the central theoretical derivation is incomplete: the IB objective of Eq. (4) is reduced in Eq. (10) to a KL regularizer without the task-relevant term, and the objective contains no mechanism that rewards transferring topology from NDiv. The reported quantitative evaluation lacks error bars and statistical tests, and no code is provided. These issues are load-bearing for the paper's main claims.","major_comments":[{"comment":"The information bottleneck objective introduced in Eq. (4) is max_Z I(Y;Z) − βI(X;Z). In Section 4.2.1, Eq. (10) replaces this with min_Z βI(R;Z), thereby dropping the task-relevant term I(Y;Z). The actual loss used in training, Eq. (17) with Eq. (18), is therefore only a KL-divergence regularizer that pushes the representation toward a standard Gaussian. It is not an information bottleneck in the sense of Eq. (4). This undermines the central claim that NOFT realizes an \"optimal-transported information bottleneck\" for content-diversity trade-off. Please justify the omission of I(Y;Z) or rename the module to reflect what is actually optimized.","section":"§4.2.1, Eq. (10) vs Eq. (4)"},{"comment":"The claimed transfer of \"local topological statistic from I2\" has no mechanism in the objective. In Eq. (11), Z = λR + (1−λ)ε, where ε is the diversity noise NDiv. Since NDiv is an i.i.d. Gaussian field (standard diffusion noise), a per-location linear combination of two independent Gaussian fields is statistically just another Gaussian field; it does not inherit spatial structure from NDiv. The Sinkhorn Attention FSA in Eq. (9) operates on NOrig alone (as NOrig + FSA(NOrig)), so there is no cross-noise alignment between NOrig and NDiv. The training losses in Eqs. (15), (17), and (18) contain no term that rewards or measures information from NDiv. The ablation in Figure 7(a) shows only that FSA reshapes the source noise; it does not establish that NDiv's spatial pattern is transferred. As written, the observed diversity is explainable as ordinary stochastic perturbation around NOrig. Please provide a derivation or a controlled experiment isolating the transfer mechanism.","section":"§4.2.1, Eq. (11) and Eq. (9)"},{"comment":"The quantitative comparison in Table 1 reports single-run numbers with no error bars, confidence intervals, or significance tests. The claimed superiority on self-sim (0.038 vs. 0.045) and DINO-I (0.841 vs. 0.753) could be within run-to-run noise. The user study uses only 10 users with 20 samples per method, and no inter-subject agreement or test-retest reliability is reported. Without statistical grounding, the claim of \"consistent superiority\" is not substantiated. Please report variance across multiple seeds/runs, perform significance tests, and release code and data to support reproducibility.","section":"Table 1 and user study"}],"minor_comments":[{"comment":"There is a typo in Eq. (3): \"NOirg\" should be \"NOrig\". In Eq. (15), the target is written as XOrig but should be NOrig based on the context.","section":"Eq. (3) and Eq. (15)"},{"comment":"The notation \"log(1−λ)^2\" is ambiguous; it should be written as log((1−λ)^2) to avoid confusion with (log(1−λ))^2.","section":"Eq. (16) and Eq. (17)"},{"comment":"The Sinkhorn scalars α and β in Eq. (14) are not explicitly defined in terms of the iterative normalization steps of Algorithm 2; please clarify the connection between the transport plan formula and the algorithm.","section":"Eq. (14) and Algorithm 2"},{"comment":"The user study is described as involving only 10 users; please report a power analysis or inter-subject agreement to justify the sample size.","section":"§5, user study"},{"comment":"The L1 values in Figure 7 are not defined in the main text; please specify what L1 measures and how it is computed to allow comparison with Table 1.","section":"Figure 7"},{"comment":"The statement that NOFT \"completely decouples\" from external control signals is overstated for the referenced variant, which explicitly uses IP-Adapter features and diffusion inversion of IRef.","section":"§4.1.1, Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The paper is squarely within the journal's scope, but the central theoretical claim needs substantial revision. The missing I(Y;Z) term and the lack of a mechanism for NDiv transfer are not merely presentational issues; they affect the core contribution. I believe the authors could potentially address this by reformulating the objective (e.g., adding a term that explicitly rewards mutual information with NDiv or a feature-level similarity with I2) and by providing a controlled experiment that isolates whether NDiv's structure is transferred. However, if the method is simply stochastic interpolation, the contribution is much weaker. Also, the absence of code and error bars is a concern for reproducibility. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read on NOFT (arXiv:2505.12235).\n\nThe genuinely new thing here is the operational move: treat the noise latent as a controllable workbench, train a 14K-parameter Sinkhorn-attention + information-bottleneck module that nudges a source noise with a second diversity noise, and apply it plug-and-play to 2D and 3D generation. That framing is not in the prior work I've seen, and the practical appeal is real—no per-concept fine-tuning, no external control maps, just a few minutes of test-time training. The qualitative results look strong, especially for a method that uses only noise manipulation, and the 3D extension is a nice addition.\n\nThe soft spots are real but mostly fixable. The load-bearing claim—that NOFT transfers \"local topological statistics\" from the second noise's image—is not supported by the equations. In Sec. 4.2.1, the IB objective starts as max I(Y;Z) − βI(X;Z) in Eq. 4, but Eq. 10 drops the task-relevant term I(Y;Z). That leaves only βI(R;Z), which shrinks information between the representation and the bottleneck, and the actual training loss in Eq. 15 is just MSE against the source noise. There is no term that rewards, measures, or even touches information from the diversity noise NDiv. NDiv enters only as the independent Gaussian ε in Eq. 11, mixed per-location with λR + (1−λ)ε. With both R and ε i.i.d. Gaussian, the output is statistically another Gaussian sample. The Sinkhorn attention operates only on NOrig, so there's no cross-noise alignment anywhere. I agree with the stress-test note: the objective never rewards transfer from NDiv, so the observed diversity is equally explainable as ordinary stochastic variation around NOrig.\n\nThe missing experiment that would settle this is direct noise interpolation—just denoise λN1 + (1−λ)N2 and compare to NOFT. If the interpolation baseline gives similar diversity and fidelity, then the IB/SA machinery isn't doing what the paper claims. Also, the quantitative table has no error bars, the user study is only 10 people, and no code is released. These are all addressable in revision. The citation pattern looks fair; the self-cite to [31] for the IBA-style bottleneck is legitimate, not an inflation.\n\nOn balance, I'd send this to peer review, not desk reject. The idea is novel and the empirical results are promising enough to deserve referee time. But the authors need to either fix the derivation or reframe the claim as an empirical heuristic, add the interpolation baseline, report variance, and release code. Right now the central claim is plausible but unsupported by the presented math. This is a paper for people working on test-time adaptation and controllable generation; if the results hold, it's a useful plug-in.","headline":"A promising plug-and-play noise-editing idea whose central transfer mechanism is missing from its own equations — worth a serious revision.","tokens_in":12972,"tokens_out":3931,"would_cite":false,"duration_ms":34548,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"NOFT fine-tunes the noise latent of a frozen diffusion model to generate highly correlated yet diverse 2D and 3D assets, using only about 14K trainable parameters and roughly 10 minutes of training.","keywords":["noise finetune","information bottleneck","Sinkhorn optimal transport","diffusion model","controllable generation","asset creation","2D and 3D generation"],"falsifier":"Run a controlled experiment where the diversity noise's spatial layout is scrambled (e.g., permute its patches or reverse its channels) and observe whether NOFT still transfers the local object parts, such as the head of a cat in a cat-shaped cake. If local topology transfer persists despite scrambling, the claim that noise locally encodes image topology is weakened; if it breaks, the claim is supported. Additionally, compare NOFT against a variant with the Sinkhorn attention removed while keeping the same bottleneck—if the local structure fidelity metrics (DINO self-similarity, DINO-I) do not drop, the Sinkhorn attention's role in contextual preservation is disconfirmed.","tokens_in":11829,"feed_emoji":"🎨","tokens_out":2161,"duration_ms":23461,"temperature":0.7,"pith_summary":"This paper argues that the initial Gaussian noise of a diffusion model is not random after all: it implicitly carries the topological and textural manifold of the image to be generated. The authors introduce NOFT, a plug-and-play module that fine-tunes this noise at test time through an optimal-transported information bottleneck, trading off content preservation against diversity without any external control signals or fine-tuning of the base generator. A sympathetic reader should care because it suggests a general, lightweight way to reimagine assets across 2D image and 3D model generators, using the noise itself as the control surface.","feed_headline":"A 14K-parameter noise edit adds diversity without losing content","feed_subtitle":"Fine-tune the diffusion noise, not the model, to reimagine 2D and 3D assets with preserved structure and texture.","key_machinery":"The central object is the Optimal-Transported Information Bottleneck (OTIB) module, which combines a Sinkhorn-Attention (doubly-stochastic attention derived from optimal transport) with a noise-level information bottleneck filter λ. The bottleneck mixes the source noise with a random Gaussian noise via Z = λR + (1−λ)ε, where λ is learned, and the Sinkhorn attention provides spatially aware transport that preserves local structure and appearance. This module operates entirely on noise tensors, so it can be plugged into frozen 2D latent diffusion and 3D structured latent generators without altering their forward or denoising processes.","core_discovery":"The central claim is that the contextual information needed to control structure and appearance already lives inside the noise latent of a pre-trained diffusion model, and that a small trained perturbation of that noise—rather than an explicit adapter, feature manipulation, or model fine-tuning—can produce images that preserve the global content of the source while visibly varying its local topology and texture. NOFT learns this perturbation by compressing the source noise and injecting information from a second diversity noise, guided by an information bottleneck and a Sinkhorn-optimal-transport attention module, trained with a pixel-level reconstruction loss and a manifold-level KL compression loss.","pith_inferences":["A testable extension would be to apply NOFT's noise-mixing operation to other generative models with compressed latents (e.g., autoregressive image models or video diffusion) and measure whether the same topology-transfer effect appears, which would support the paper's stronger claim that noise latents universally encode scene manifolds.","The method implicitly assumes that the local spatial statistics of a noise patch correspond to local image features; one could isolate this by ablating the spatial order of the diversity noise (e.g., shuffling patches) and checking whether local structure transfer degrades, which the paper does not explicitly test.","If noise finetuning is indeed a sufficient control surface, it suggests a new class of lightweight personalization tools that operate before generation rather than during it, possibly enabling real-time interactive asset editing on consumer hardware."],"forward_implications":["If correct, any pre-trained diffusion generator could be equipped with a lightweight noise-fineting module to produce content-faithful variations, removing the need for control signals such as Canny edges, depth maps, or reference-image adapters.","The approach extends beyond 2D images to 3D asset generation, as demonstrated by applying NOFT to the structured latent space of TRELLIS, suggesting that noise-level editing is a generator-agnostic interface.","The trade-off weight β directly controls the degree of diversity, giving practitioners a single knob to balance content preservation against variation, from mild tweaks to substantial structural and semantic changes.","Because NOFT trains on random noise pairs alone, it could be adapted to new generators or domains with minimal data and compute, potentially enabling fast on-the-fly customization for asset pipelines."],"supporting_citations":[{"why":"Provides the frozen latent diffusion generator (Stable Diffusion) that NOFT operates on, defining the noise latent space and the denoising procedure.","marker":"[6]"},{"why":"Introduces the information bottleneck principle that motivates the compression term in NOFT's trade-off between content preservation and diversity.","marker":"[15]"},{"why":"Supplies the Sinkhorn algorithm for entropy-regularized optimal transport, which underlies the Sinkhorn attention module used for local structure preservation.","marker":"[17]"},{"why":"Provides the multi-prompt Sinkhorn attention concept that NOFT's Sinkhorn-Attention forward pass builds upon for spatially aligned feature transport.","marker":"[18]"},{"why":"Establishes the information-bottleneck attribution formulation and the KL-divergence approximation that NOFT adapts to noise-level manifolds.","marker":"[16]"},{"why":"Supplies the null-text inversion method used to recover the contextual latent (inverse noise) for referenced NOFT_2D variants.","marker":"[29]"},{"why":"Provides the structured 3D latent generator (TRELLIS) that NOFT_3D applies to, demonstrating extension beyond 2D image generators.","marker":"[8]"},{"why":"Offers the IP-Adapter used for appearance transfer in referenced NOFT_2D, providing the image-prompt feature extraction that NOFT perturbs.","marker":"[5]"}],"fun_headline_variants":["Fine-tune noise, not model: 14K params for diverse images","Test-time noise finetune: 14K params, content-preserved diversity","NOFT: Noise finetune with info bottleneck for correlated assets","Edit the noise, keep the content: 14K params for new views","A 14K-param noise tweak for content-loyal diversity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that simply mixing noise values at matching positions between the source and diversity noises, without aligning their spatial structure, is enough to transfer the local topological character of the diversity image into the generated output; if this per-location mixing does not actually carry local structure, the central claim of topology-preserving variation collapses.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tune noise, not model: 14K params for diverse images","Test-time noise finetune: 14K params, content-preserved diversity","NOFT: Noise finetune with info bottleneck for correlated assets","Edit the noise, keep the content: 14K params for new views","A 14K-param noise tweak for content-loyal diversity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2896,"prompt_tokens":922,"completion_tokens":1974,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":1875}},"tokens_in":538,"tokens_out":1974,"duration_ms":12591,"temperature":1.0,"reasoning_tokens":1875,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:38:46.287018+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled experiment where the diversity noise's spatial layout is scrambled (e.g., permute its patches or reverse its channels) and observe whether NOFT still transfers the local object parts, such as the head of a cat in a cat-shaped cake. If local topology transfer persists despite scrambling, the claim that noise locally encodes image topology is weakened; if it breaks, the claim is supported. Additionally, compare NOFT against a variant with the Sinkhorn attention removed while keeping the same bottleneck—if the local structure fidelity metrics (DINO self-similarity, DINO-I) do not drop, the Sinkhorn attention's role in contextual preservation is disconfirmed.","supporting_citations":[{"cited_title":"Deep learning and the information bottleneck principle,","cited_arxiv_id":null,"evidence_quote":"Introduces the information bottleneck principle that motivates the compression term in NOFT's trade-off between content preservation and diversity."},{"cited_title":"Sinkhorn distances: Lightspeed computation of optimal transport,","cited_arxiv_id":null,"evidence_quote":"Supplies the Sinkhorn algorithm for entropy-regularized optimal transport, which underlies the Sinkhorn attention module used for local structure preservation."},{"cited_title":"Otseg: Multi-prompt sinkhorn attention for zero-shot semantic segmentation,","cited_arxiv_id":null,"evidence_quote":"Provides the multi-prompt Sinkhorn attention concept that NOFT's Sinkhorn-Attention forward pass builds upon for spatially aligned feature transport."},{"cited_title":"Restricting the flow: Information bottlenecks for attribution,","cited_arxiv_id":null,"evidence_quote":"Establishes the information-bottleneck attribution formulation and the KL-divergence approximation that NOFT adapts to noise-level manifolds."},{"cited_title":"Null-text inversion for editing real images using guided diffusion models,","cited_arxiv_id":null,"evidence_quote":"Supplies the null-text inversion method used to recover the contextual latent (inverse noise) for referenced NOFT_2D variants."},{"cited_title":"Structured 3d latents for scalable and versatile 3d generation,","cited_arxiv_id":null,"evidence_quote":"Provides the structured 3D latent generator (TRELLIS) that NOFT_3D applies to, demonstrating extension beyond 2D image generators."}],"review_version":1}