{"id":"2aa5c699-38ce-46d9-9566-190654874de3","arxiv_id":"2506.16889","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ITO-Master optimizes the reference embedding during inference to give users finer control over mastering style transfer in both black-box and white-box audio processors.","lead":"A new mastering system lets users fine-tune how a reference track's style is applied by optimizing the reference's learned audio embedding at inference time. The approach improves style matching and listener similarity ratings, though it can slightly reduce audio naturalness.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The AF/DRV gains after ITO are partly circular because the ITO objective (§3.4) and the primary evaluation metric (§4.3) are the same AF loss; FAD mostly worsens and Table 1 lacks error bars, so the central claim of improved mastering performance is not yet established.","rationale":"The central claim requires not just that AF/DRV numbers improve, but that ITO provides perceptually meaningful refinement of mastering style. Since AF loss is both the optimized objective and the primary evaluation metric, the Table 1 AF gains are expected and partly tautological. The independent FAD metric mostly worsens after ITO, a tradeoff acknowledged in prose but omitted from the abstract, which weakens the claim as stated. The subjective test provides real support, but its ITO-specific comparison is not quantified in the text (no mean scores, CIs, or per-condition p-values); the sentence 'further enhanced by ITO' follows a global pairwise t-test that may not isolate ITO. Therefore, the reader's conditional verdict is appropriate. I do not see a reason to move to reject: the framework is novel, the white-box/black-box comparison is informative, code is released, and text-prompt ITO is a promising application. But until the independent subjective or objective evidence for the specific ITO increment is reported with uncertainty, the strongest claim should remain conditional.","tokens_in":10477,"tokens_out":7315,"duration_ms":83985,"concrete_test":"Re-analyze the existing MUSHRA data, restricting to the matched +ITO versus no-ITO pairs for the same model and input/reference pairs, and run a paired bootstrap or Wilcoxon test; report the mean similarity difference and 95% confidence interval. If the interval includes zero, the claim that ITO improves mastering similarity is not supported by the subjective test. As a complementary check, compute a held-out style embedding (e.g., a pretrained FXencoder not used in training or ITO) before and after ITO; a null result would confirm that the AF/DRV gains are optimization artifacts rather than evidence of improved style transfer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the claim that ITO improves mastering style transfer, the paper uses AF loss both as the ITO objective (Section 3.4) and as the headline metric (Section 4.3, Table 1). Any gradient-based optimization will reduce its own objective, so the AF drop from 0.125/0.186 to 0.099/0.139 after ITO is expected rather than evidence of perceptual style improvement. The independent FAD metric worsens in most rows (e.g., Black-box + trainPhi FAD_CLAP 159.8 to 182.2; White-box + trainPhi FAD_CLAP 93.2 to 105.2), a tradeoff acknowledged in prose but omitted from the abstract. Table 1 reports point estimates without confidence intervals or significance tests. The subjective listening test partly breaks the circularity, but the ITO-vs-no-ITO contrast is not reported with effect sizes or per-condition statistics, so the claim that ITO specifically improves mastering similarity remains under-supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ITO-Master, an inference-time optimization framework for reference-based music mastering style transfer. It trains a Master Style Converter Ψ in self-supervised fashion with random FX manipulations, and at inference time optimizes the reference embedding zref using an auxiliary content-independent Audio Feature (AF) loss while keeping Ψ fixed. The converter is instantiated in both black-box and white-box forms, the latter using a differentiable six-module mastering chain. Experiments on MTG-Jamendo compare against feature-matching and E2E baselines, reporting AF loss, DRV, cosine similarity, and FAD, plus a 10-participant MUSHRA-type listening test and a qualitative text-prompt study. The central claim is that ITO improves mastering style similarity and provides user-controllable refinement without retraining.","tokens_in":10674,"tokens_out":4157,"duration_ms":44451,"significance":"If the central claim were fully supported, ITO-Master would be a useful contribution to the audio-effects style transfer literature: it introduces a clean inference-time mechanism for user control, systematically compares black-box and white-box converter designs, and ships an open-source differentiable mastering chain with a demo. The qualitative text-conditioned ITO using CLAP embeddings is a promising creative tool. However, the primary objective evidence is weakened by the fact that the AF loss is both the ITO objective and the headline metric, by the absence of error bars or significance tests in Table 1, and by the generally worsening FAD scores. The DRV improvements and the subjective listening test provide independent support, but as reported they are not sufficient to fully establish the paper's claim that ITO 'improves mastering performance across different styles.'","major_comments":[{"comment":"The AF loss is used both as the optimization objective for ITO (Section 3.4) and as the primary objective evaluation metric (Section 4.3, Table 1). Consequently, the reported AF improvements after ITO (e.g., Black-box + trainPhi: 0.125 to 0.099; White-box + trainPhi: 0.186 to 0.139) are expected by construction for any gradient-based optimizer and cannot serve as independent evidence that ITO improves mastering style transfer. The paper should either report AF loss only as a sanity check of the optimization and rely on non-optimized metrics for the central claim, or provide an independent, held-out style metric that is not part of the ITO objective.","section":"§3.4 and §4.3, Table 1"},{"comment":"All objective results in Table 1 are reported as point estimates with no confidence intervals, error bars, or significance tests. Several ITO-vs-no-ITO differences are small (e.g., DRV 0.577 vs. 0.567; cos sim 0.945 vs. 0.946), and without uncertainty estimates it is impossible to tell whether the improvements are reliable. The authors should report repeated-run or bootstrap intervals and perform significance tests for each metric and condition, especially for DRV and cosine similarity.","section":"Table 1 and §5.1"},{"comment":"The listening test is described as showing 'significant improvements (pairwise t-test, p<0.05)', but the manuscript does not state which pairwise comparisons were tested, whether multiple-comparison correction was applied, or what the effect sizes and per-condition means/confidence intervals were. Moreover, the MUSHRA-type design lacks a hidden reference and a high anchor, making the absolute similarity scores difficult to interpret. The authors should report the full per-condition statistics, specify the exact comparisons, and discuss the impact of the missing anchors on the validity of the subjective claim.","section":"§5.2, Subjective Evaluation"},{"comment":"The paper optimizes the raw 2048-dimensional reference embedding zref with no regularization or manifold constraint, yet claims this enables 'micro-level adjustments'. The FAD deterioration observed in most rows (e.g., Black-box + trainPhi FAD_CLAP 159.8 to 182.2; White-box + trainPhi FAD_CLAP 93.2 to 105.2) suggests that the optimized embedding may drift outside the distribution the converter was trained on. The authors should analyze embedding drift (e.g., distance from the original zref, distribution statistics before/after ITO) or add a regularization term, to support the claim that ITO makes only micro-level changes rather than arbitrary perturbations.","section":"§3.4, Figure 1(b), §5.1"}],"minor_comments":[{"comment":"The title in the submitted text contains an odd line-break artifact: 'INFERENCE-TIME OPTIMIZA TION' should read 'INFERENCE-TIME OPTIMIZATION'.","section":"Title"},{"comment":"The evaluation setup says 200 songs are selected from MTG-Jamendo, with 100 as xin and 100 as xref, but it is not stated whether the two sets are disjoint and how the xin/xref pairings are formed. Please clarify to avoid potential content or genre leakage.","section":"§4.1"},{"comment":"The abstract states that ITO 'improves mastering performance', but the FAD trade-off is acknowledged only in Section 5.1 prose and is omitted from the abstract. The authors should qualify the abstract claim to reflect the naturalness cost.","section":"Abstract and §5.1"},{"comment":"The qualitative text-prompt analysis is based on a single instrumental track and one run per prompt; this is fine as a qualitative demonstration, but the manuscript should explicitly label it as anecdotal rather than as evidence of general text-guided control.","section":"§5.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of the conference and the core idea is interesting, but the current evaluation does not yet support the central claim because the main optimized metric is also the primary evaluation metric. This is fixable with additional analysis and statistics, so I recommend major revision rather than rejection. I would also encourage the authors to make the subjective-test data and evaluation scripts available to support reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper is a legitimate new application of inference-time optimization, but the main objective result is weaker than the abstract implies. The core idea—optimizing the reference embedding zref at inference rather than processor parameters or diffusion latents—is genuinely new in the mastering-style-transfer setting, and the black-box vs white-box comparison is useful. The authors also ship code and a demo, which I value.\n\nWhat it does well: the differentiable mastering chain with six modules is a real step toward realistic white-box processing. The subjective listening test, though small (10 experts), points in the same direction as the stated claims and gives some independent support. The qualitative text-prompt experiments are a nice extra, showing the ITO mechanism can steer toward genre-like characteristics. The paper is clearly written and the related work is honestly situated—ST-ITO optimizes parameters, DITTO optimizes latents, and this paper optimizes the reference embedding, so the novelty claim is fair.\n\nWhere I'd push back: the AF loss is used both as the ITO objective (Section 3.4) and as the primary evaluation metric (Section 4.3). Any gradient-based optimizer will reduce its own objective, so the AF drop from 0.125/0.186 to 0.099/0.139 is expected, not evidence of perceptual improvement. The independent FAD metric mostly worsens after ITO (e.g., 159.8 to 182.2 for black-box with trained encoder; 93.2 to 105.2 for white-box), and the abstract omits this trade-off. Table 1 also has no error bars or significance tests, so some of the differences may be noise. The subjective test partially breaks the circularity, but the paper doesn't report effect sizes or per-condition statistics for ITO versus no-ITO.\n\nThat said, I don't think the central idea collapses. ITO on zref is plausible, the white-box interpretability angle is genuinely useful, and the authors acknowledge the FAD trade-off in the body. The main fix is evaluative hygiene: separate the optimization objective from the headline metric, report uncertainty, and put the trade-off in the abstract. The conclusion that ITO helps reference-based mastering refinement is directionally supported, not fully proven.\n\nWho's this for? People working in audio effects modeling, automatic mastering, and anytime style-transfer methods. It's a solid ISMIR-level contribution, not a breakthrough, but it deserves proper peer review.\n\nMy recommendation: send it to review, but with the expectation that the circular metric gap and missing uncertainty quantification be addressed before acceptance.\n\nBest,\n[You]","headline":"A credible new application of inference-time optimization to mastering style transfer, but the headline AF gains are partly circular because the same loss drives optimization and evaluation; still worth a careful peer review.","tokens_in":11230,"tokens_out":1099,"would_cite":true,"duration_ms":13426,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimizing the reference embedding at inference time improves reference-based mastering style transfer and gives users finer control.","keywords":["music mastering","style transfer","inference-time optimization","reference embedding","differentiable audio effects","audio feature loss","text-guided audio processing"],"falsifier":"A decisive check would be to run ITO for a fixed input and reference over a grid of optimization steps and, with expert listeners in a blind test, determine whether the step count that minimizes the AF loss is also the one judged closest to the reference; if AF loss keeps decreasing while perceptual similarity does not increase, the auxiliary objective is not tracking the claimed mastering style.","tokens_in":10265,"feed_emoji":"🎛️","tokens_out":14046,"duration_ms":128664,"temperature":0.7,"pith_summary":"This paper is trying to establish that a music mastering style-transfer system can be made more controllable and more accurate by optimizing the reference embedding at inference time, after training is complete. The proposed framework, ITO-Master, keeps the trained style converter fixed and adjusts a 2048-dimensional style vector through gradient descent on an audio-feature loss that is meant to be independent of the musical content. If the claim holds, users of an automatic mastering system would be able to fine-tune a transfer toward the target style, or toward a text description of a genre, without retraining the model or hand-adjusting dozens of effect parameters. The paper reports that this inference-time optimization lowers the audio-feature loss and dynamic-range variability, and that expert listeners rate the ITO outputs as closer to the reference mastering style, though the naturalness metrics it reports worsen after optimization.","feed_headline":"A reference-embedding tweak refines mastering style at inference","feed_subtitle":"Fine-tuning the reference embedding improves style match and adds text control without retraining.","key_machinery":"The mechanism that carries the argument is Inference-Time Optimization on the reference embedding: at inference, the style vector $z_{\\mathrm{ref}}$ is treated as a free variable and updated by gradient descent under an auxiliary loss $\\mathcal{L}_{\\mathrm{aux}}$ while the pre-trained converter is frozen. The auxiliary loss is the Audio Feature (AF) loss, a weighted combination of audio-feature transformations intended to capture dynamics, spatialization, and spectral character without being tied to the musical content. On the white-box side, the differentiable mastering chain—six effect modules ordered as EQ, distortion, three-band compressor, makeup gain, stereo imager, and limiter, with differentiable all-pole filters in the compressor and limiter—is what makes the parameters interpretable and lets the effect of the optimized embedding be observed.","core_discovery":"The central claim is that the reference embedding $z_{\\mathrm{ref}}$—the representation of the reference track that conditions the mastering converter—is the right place to apply inference-time optimization for style transfer. In both a black-box waveform converter and a white-box differentiable mastering chain, updating $z_{\\mathrm{ref}}$ by backpropagating the Audio Feature loss for up to 100 steps improves the objective metrics: the black-box model's AF loss falls from 0.125 to 0.099 and dynamic-range variability from 0.577 to 0.567, while the white-box model's AF loss falls from 0.186 to 0.139 and dynamic-range variability from 0.521 to 0.474. Cosine similarity between the output and reference FX embeddings stays roughly constant, and listening-test scores improve with ITO. The same optimization applied to the white-box chain's 46 parameters instead of the embedding performs worse on every metric, suggesting that the embedding is a more effective optimization target than the processor parameters when the loss is content-independent.","pith_inferences":["Because ITO touches only the reference embedding, the same frozen converter could be steered by other differentiable objectives, such as a target loudness, a target stereo width, or a perceptual quality penalty, without any retraining; the paper demonstrates only an audio-feature objective and a text-audio objective.","The reported naturalness degradation after ITO suggests a multi-objective variant that adds a quality penalty to the auxiliary loss might recover the style gain while protecting audio quality, a combination the paper does not test.","The text-prompt steering experiment indicates the method is a generic steering mechanism for differentiable effect chains, so it should transfer to other audio effects such as reverb or EQ matching, even though the paper only studies mastering processors."],"forward_implications":["If ITO-Master is right, mastering style transfer can be refined after the initial inference, so a single trained model can produce many variants of one transfer from the same reference simply by choosing how long to optimize the embedding.","Optimizing the reference embedding with a content-independent loss is more efficient and more effective than optimizing the whole differentiable effect chain, which the paper's experiments show degrades all metrics.","Training the reference encoder together with the converter is a necessary ingredient: with a fixed pretrained encoder the model performs worse, while joint training improves both AF loss and dynamic-range variability.","The ITO objective is not tied to mastering features; the paper's text-prompt experiment shows the same embedding optimization can steer the output toward genre descriptions, so the mechanism acts as a flexible creative control.","Because the reported naturalness metrics worsen after ITO, the number of optimization steps is a practical tuning knob, and the style/naturalness trade-off must be managed in deployment."],"supporting_citations":[{"why":"Defines the Audio Feature loss that serves as both the ITO objective and the main style-similarity metric.","marker":"[13]"},{"why":"Supplies the pretrained reference encoder used to produce the embedding and to compute cosine-similarity evaluation.","marker":"[12]"},{"why":"Provides the end-to-end remastering baseline that the proposed black-box converter extends and is compared against.","marker":"[6]"},{"why":"Defines the normalization-based feature-matching baseline whose statistics the training pipeline uses.","marker":"[19]"},{"why":"Provides one of the feature-matching baseline systems used in the comparison.","marker":"[7]"},{"why":"Establishes the prior use of inference-time optimization for audio effects style transfer that this work adapts to mastering.","marker":"[17]"},{"why":"Makes the compressor and limiter differentiable, which is what allows the white-box mastering chain to model time-varying dynamics.","marker":"[21]"},{"why":"Provides the text-audio embeddings used in the text-prompt ITO experiment.","marker":"[22]"},{"why":"Defines FAD, the naturalness metric used to measure acoustic quality of the generated outputs.","marker":"[28]"}],"fun_headline_variants":["Embedding tweak boosts music mastering style at inference","Inference-time embedding optimization sharpens mastering style","Optimizing reference embedding refines audio mastering on the fly","Better mastering style: tune the reference embedding, not the knobs","Reference-embedding optimization gives finer mastering control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Audio Feature loss measures mastering style independently of musical content, so that minimizing it at inference time moves the output toward the reference's mastering style rather than toward some unrelated artifact.","fun_headline_variants_meta":{"raw":{"variants":["Embedding tweak boosts music mastering style at inference","Inference-time embedding optimization sharpens mastering style","Optimizing reference embedding refines audio mastering on the fly","Better mastering style: tune the reference embedding, not the knobs","Reference-embedding optimization gives finer mastering control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000544,"raw_usage":{"total_tokens":2598,"prompt_tokens":936,"completion_tokens":1662,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":1584}},"tokens_in":552,"tokens_out":1662,"duration_ms":11503,"temperature":1.0,"reasoning_tokens":1584,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:16:28.496611+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check would be to run ITO for a fixed input and reference over a grid of optimization steps and, with expert listeners in a blind test, determine whether the step count that minimizes the AF loss is also the one judged closest to the reference; if AF loss keeps decreasing while perceptual similarity does not increase, the auxiliary objective is not tracking the claimed mastering style.","supporting_citations":[{"cited_title":"End-to-end music re- mastering system using self-supervised and adversarial training,","cited_arxiv_id":null,"evidence_quote":"Defines the Audio Feature loss that serves as both the ITO objective and the main style-similarity metric."},{"cited_title":"Differentiable signal processing with black- box audio effects,","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained reference encoder used to produce the embedding and to compute cosine-similarity evaluation."},{"cited_title":"Classic Music","cited_arxiv_id":null,"evidence_quote":"Provides the end-to-end remastering baseline that the proposed black-box converter extends and is compared against."},{"cited_title":"Music mixing style transfer: A contrastive learning approach to disentangle audio effects,","cited_arxiv_id":null,"evidence_quote":"Defines the normalization-based feature-matching baseline whose statistics the training pipeline uses."},{"cited_title":"Our experiments showed that training the reference encoder Φ alongside Ψ improves performance","cited_arxiv_id":null,"evidence_quote":"Provides one of the feature-matching baseline systems used in the comparison."},{"cited_title":"Blind estima- tion of audio processing graph,","cited_arxiv_id":null,"evidence_quote":"Establishes the prior use of inference-time optimization for audio effects style transfer that this work adapts to mastering."},{"cited_title":"Towards zero-shot amplifier modeling: One-to-many amplifier modeling via tone embedding control,","cited_arxiv_id":null,"evidence_quote":"Makes the compressor and limiter differentiable, which is what allows the white-box mastering chain to model time-varying dynamics."},{"cited_title":"DITTO: Diffusion inference-time t- optimization for music generation,","cited_arxiv_id":null,"evidence_quote":"Provides the text-audio embeddings used in the text-prompt ITO experiment."},{"cited_title":"Differentiable all-pole filters for time- varying audio systems,","cited_arxiv_id":null,"evidence_quote":"Defines FAD, the naturalness metric used to measure acoustic quality of the generated outputs."}],"review_version":2}