{"id":"5e38b3ff-2027-48af-a6ed-3dcfaac68d6f","arxiv_id":"2507.12773","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A Gaussian-process optimizer that mixes whole-filter ratings with a few oracle-supplied optimal-coordinate values improves audio personalization in simulations and a small user study, though the core assumptions remain untested.","lead":"Researchers combine two ways of asking a listener what sounds good: rating whole audio clips and telling the system the exact gain they want at individual frequencies. In a three-person test the hybrid method beat an audiogram-only baseline within 30 questions, but it relies on an idealized oracle and no convergence proof.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real audio experiments feed the optimizer exact entries of the simulated corrupting filter, so the oracle assumption is never tested; with realistic noisy dimension queries the hybrid advantage may vanish.","rationale":"The reader's weakest assumption correctly identifies the oracle as a weak point. I partially agree: I sharpen it by pointing out that the real-world experiment not only assumes an oracle but actively uses the exact ground-truth corrupting filter as the dimension queries, so the motivating application is never tested with measurements a user could actually produce. The missing filter-query-only baseline in the real experiments is an additional gap: without ALEBO(L=0) under the same total budget, the 'hybrid beats either alone' claim rests solely on synthetic results. These synthetic results do provide some support for the hybrid idea under an exact oracle, which is why I do not move to REJECT; the reader's CONDITIONAL verdict is appropriate, with the condition that the authors demonstrate robustness to realistic dimension-query noise and include a Qf-only baseline. This is consistent with good-faith reading: the paper is transparent about the oracle assumption and lists independence as open work, but the empirical evidence for the central application claim is not yet sufficient.","tokens_in":14123,"tokens_out":6732,"duration_ms":80297,"concrete_test":"Run the random-distortion audio experiment (or its exact simulation, N=4000, L=5, B=30) twice: once with oracle dimension queries h*[j] as in the paper, and once with noisy queries h*[j]+δ_j, δ_j drawn from N(0,5^2) dB and quantized to 5 dB steps; include an ALEBO(L=0) filter-query-only baseline in both. If ORACLE BO no longer beats the filter-query-only baseline under noisy dimension queries, the oracle assumption is load-bearing and the audio-personalization claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is that the motivating application is validated under an oracle that supplies exact h*[j], but real users cannot provide those values, and the paper's own experiments sidestep the problem. In the audio experiment, the dimension queries are taken directly from the corrupting filter used to generate the audio: for the random-distortion condition the appendix states 'we generate a random corrupting filter b2,' and the Qd values are the values of that filter at 7 frequencies. These are exact entries of the known h* used to synthesize the task, not measurements a user can give. An actual audiogram measures detection thresholds at a few frequencies with coarse steps; it does not reveal the satisfaction-maximizing gain h*[j], and interactions across frequencies (which the paper lists as an open problem in its own Follow-up Work) mean Eq. 20's factorized Gaussian G_j(N(μ=h*[j], σ)) is built on an independence assumption that is false in hearing. Because DMS selects the next filter by matching candidates to these oracle values, any error in h*[j] propagates directly into the acquisition. No sensitivity analysis to measurement error is reported. The central claim of the paper is about real audio personalization, yet the real-world experiment has not been run with realistic dimension-query noise; the 3.3-point improvement is measured against the coarse audiogram interpolation baseline, not against a pure filter-query BO method (ALEBO L=0) under the same budget, which is the appropriate test of 'hybrid outperforms either type alone.'","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ORACLE BO, a hybrid querying approach for sample-constrained black-box optimization. In addition to standard filter queries Qf (full evaluations of the unknown satisfaction function), the method assumes an Oracle that can return exact coordinates h*[j] of the true minimizer h*. Building on ALEBO's sparse Gaussian-process Bayesian optimization, the method adds two modules: a Batch Acquisition Function (BAF) that produces q candidate filters in a random embedding via q-Expected Improvement, and a Dimension Matched Sampler (DMS) that selects among these candidates by matching them, through a factorized Gaussian likelihood (Eq. 20), to the oracle coordinates. The paper reports synthetic experiments on staircase satisfaction functions and claims to evaluate Branin, Hartmann6, and Rosenbrock benchmarks, plus a real audio personalization study with three volunteers in which ORACLE BO is compared against a coarse audiogram interpolation baseline. The authors explicitly state that they lack a mathematical proof and rely on empirical results, reporting an average 3.3-point satisfaction improvement within a budget of B=30 queries.","tokens_in":14446,"tokens_out":6055,"duration_ms":72201,"significance":"The hybrid-query formulation is a genuinely novel idea: mixing coarse, high-information oracle queries with full-objective filter queries is natural for human-in-the-loop settings and could open useful directions in Bayesian optimization. The paper is clearly written and transparent about its lack of an analytical convergence guarantee, and it ships a demo website. However, the empirical support for the central claim is narrow: the synthetic results shown are limited to one staircase function family (P1), the promised benchmark results are absent, the real-user study has N=3 with no error bars, and—most importantly—the audio experiments use oracle coordinates that are exact entries of the known corrupting filter, so the motivating application is never tested under the realistic condition that dimension queries are noisy or approximate. The idea is promising but the current evidence does not establish that hybrid querying outperforms a pure filter-query baseline in real audio personalization.","major_comments":[{"comment":"The central motivating application is validated under an oracle that supplies exact coordinates h*[j]. The appendix states that, for the random-distortion condition, 'we generate a random corrupting filter b2' and the Qd values are the values of that filter at 7 frequencies—these are exact entries of the known h* used to synthesize the task, not measurements a user can provide. In contrast, a clinical audiogram measures hearing thresholds, not the satisfaction-maximizing gain h*[j], and the paper itself lists correlated/non-independent dimension queries as an open problem. Because Eq. 20 centers a Gaussian at h*[j] and multiplies this factor into the acquisition, any error or interaction in those dimension queries propagates directly into the chosen filter. No sensitivity analysis to dimension-query noise is reported. This makes the 3.3-point improvement claim ungrounded for the real audio setting.","section":"Problem Formulation and Audio Personalization Experiment Details"},{"comment":"The manuscript promises results for the benchmark functions BRANIN, HARTMANN6, and ROSENBROCK, but no such results appear in any figure or table. Figures 4–6 and Table 1 report only the staircase function P1 (the appendix defines P2 and P3, but no results are shown for them either). The abstract and introduction claim validation on 'various synthetic satisfaction functions' and 'extensive simulations,' which overstates the evidence actually provided. The benchmark results should be added, or the claims should be restricted to the staircase family.","section":"Experiment: Synthetic BlackBox Functions"},{"comment":"The real-user experiment compares ORACLE BO only against a coarse audiogram interpolation baseline, not against a pure filter-query Bayesian optimization method such as ALEBO with L=0 under the same total budget B. The central claim is that hybrid querying can outperform any single type of querying; the synthetic comparison in Figure 4 does include ALEBO(L=0), but the real audio study does not. In addition, the user study has only three volunteers, no error bars, and no repeated measures, and the raw scores in Figure 7 show large variability across users. This is too thin to support the quantitative 3.3-point claim as a general result.","section":"Audio Personalization Results, Figure 7"},{"comment":"The method introduces free parameters q (BAF batch size), sigma (DMS coordinate-matching variance), and d (embedding dimension), and Table 1 shows that regret varies dramatically with these settings—for example, at N=4000 the regret ranges from 1268 to 10213 depending on (q, sigma, d). The main figures appear to use a single favorable configuration, and the paper gives no guidance on how to choose these parameters in practice. Without a robustness analysis or a principled selection criterion, the reported gains may be specific to the chosen hyperparameters rather than inherent to the hybrid-query idea.","section":"Hyperparameter Selection, Table 1"}],"minor_comments":[{"comment":"The optimization objective is written as argmin ||f(hat h) - f(h*)||2, but f is unknown and is only accessible through queries; as written this objective cannot be evaluated. Presumably the intended statement is to minimize f(h) or the expected regret of the final recommendation.","section":"Eq. (1)"},{"comment":"The figure labels are cluttered and the score values overlap, making it difficult to verify the claimed 3.3-point improvement from the plot. The authors should provide a table of per-user scores with confidence intervals or raw data.","section":"Figure 7"},{"comment":"The text says a clinical audiogram gives 'the user's hearing loss profile h*[j]', but audiometry measures detection thresholds at discrete frequencies; the identity between thresholds and the satisfaction-maximizing filter gains is assumed without discussion. This should be clarified or corrected.","section":"Audio Personalization Experiment Details"},{"comment":"The paper alternates between minimization of f and maximization of satisfaction; for example, Eq. (1) minimizes, while the audio experiments report higher scores as better. The notation should be made consistent.","section":"Problem Formulation"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is original and worth developing, but the current evidence is substantially weaker than the claims. The absence of the promised benchmark results, the lack of a pure-filter-query baseline in the real study, and the use of exact oracle coordinates as Qd in the audio experiments are the main causes. These are fixable with additional experiments, so I recommend major revision rather than rejection, but the authors should also consider reframing the contribution as an oracle-based optimization method if realistic audio validation cannot be provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. The new thing here is the hybrid querying formulation: instead of only full-filter ratings, allow a few oracle queries that give exact coordinates of the minimizer, and fuse them through a batch acquisition function (BAF) plus a dimension-matching sampler (DMS). That combination is new relative to ALEBO and the rest of the sparse BO literature. The idea of choosing the next filter by first forming a batch of EI-optimal candidates and then selecting the one whose coordinates best match the known h*[j] values is simple and reasonable. The paper is also honest: it explicitly disclaims a proof and lists open problems, including the independence of dimension queries. Citation pattern is fine; the relevant embedding and BO papers are cited.\n\nWhere it falls short: the central empirical claim rests on an oracle that is never tested under realistic noise. In the audio experiment, the Qd values are taken directly from the corrupting filter used to generate the audio. Those are ground-truth entries of the known h*, not something a user can provide. A real audiogram measures detection thresholds at a few frequencies; it does not give the satisfaction-maximizing gain at those frequencies. Because DMS centers a Gaussian on h*[j] and selects the next filter by matching to that value, any error in the oracle answers propagates straight into the acquisition. The paper does not include a sensitivity analysis with noisy dimension queries.\n\nAlso, the real-user study has N=3, no error bars, and no pure filter-query baseline. The 3.3-point gain is measured against coarse audiogram interpolation, not against ALEBO(L=0) with the same budget, so the claim that hybrid outperforms either query type alone is not actually tested in the real experiment. The synthetic section only shows the staircase function P1; the promised Branin/Hartmann6/Rosenbrock results are absent. The hyperparameters q and sigma are swept in Table 1 but no principled selection rule is offered. No code is released.\n\nNone of this kills the idea. The contribution is real and the mechanism is plausible. But the paper overclaims the evidence. A revision that adds the missing benchmarks, a filter-only baseline, and a noisy-oracle sensitivity analysis would make it much stronger. As is, I'd still send it to referees because the hybrid querying direction is worth airing and the authors are clear about what they did and did not prove.","headline":"Hybrid filter-and-dimension querying is a genuinely new idea for BO, but the paper's evidence base is thinner than the 3.3-point headline suggests, and the oracle assumption is untested in the real audio experiments.","tokens_in":14983,"tokens_out":2251,"would_cite":true,"duration_ms":24413,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hybrid queries lift audio satisfaction by 3.3 points in 30 questions","keywords":["audio personalization","black-box optimization","Bayesian optimization","dimension queries","hybrid querying","Gaussian process regression","sparse random embeddings","hearing aid fitting"],"falsifier":"Run ORACLE BO on a synthetic staircase function while corrupting each dimension-query answer with independent noise (say, a few dB of error on the requested gain); if a few noisy hints eliminate the advantage over pure filter-query Bayesian optimization at the same budget, the claimed gain depends on exact oracle answers rather than on the hybrid structure itself.","tokens_in":13923,"feed_emoji":"🎧","tokens_out":5697,"duration_ms":60374,"temperature":0.7,"pith_summary":"This paper asks whether a fixed budget of user queries can personalize audio better if some queries are whole-filter ratings and some are pointwise hints about the ideal filter. Its answer is yes: an algorithm called ORACLE BO, built on sparse Gaussian-process regression, combines both query types and, in tests with real volunteers, reaches on average 3.3 points higher satisfaction than a strong baseline within a budget of $B=30$ queries. The paper deliberately offers empirical evidence rather than a convergence proof, and its synthetic experiments reveal a sweet spot in how much of the budget should be spent on pointwise hints.","feed_headline":"Hybrid queries lift audio satisfaction by 3.3 points in 30 questions","feed_subtitle":"Per-frequency oracle answers combined with whole-filter ratings find better personal filters within a fixed budget.","key_machinery":"The machinery is ORACLE BO's two-module extension of a sparse Gaussian-process Bayesian optimizer. After a random linear embedding reduces the high-dimensional filter space to a low-dimensional one, BAF uses MCMC slice sampling to propose a batch of jointly promising candidate filters, scored by a batch expected-improvement metric; DMS then projects the batch back to filter space and picks the candidate whose coordinates best match the available pointwise hints, using a product of Gaussians centered on $h^*[j]$. The chosen filter is the one the user rates, and the rating updates the GPR posterior; the pointwise hints steer the next batch without ever being treated as full observations of the satisfaction function.","core_discovery":"The central claim is that hybrid querying—asking a user both for overall satisfaction scores on complete filters and for the best value of individual filter coordinates—outperforms either query type alone under the same total query budget. ORACLE BO incorporates the pointwise hints through two modules: a Batch Acquisition Function (BAF), which draws a batch of candidate filters that jointly maximize expected improvement, and a Dimension Matched Sampler (DMS), which scores those candidates by a product of their acquisition value and Gaussians centered at the hinted coordinates $h^*[j]$, then queries the user on the winner. The paper reports that this produces lower regret than the ALEBO-style sparsity-based baseline on staircase and standard benchmark functions, and improves real-user satisfaction scores in audio personalization experiments.","pith_inferences":["If the advantage survives noisy pointwise hints, the same hybrid design could let users correct one coordinate of a recommended filter (e.g., 'too much salt') instead of rating entire alternatives from scratch.","The sweet-spot observation suggests an adaptive controller that spends dimension queries early and then switches to filter queries once the posterior has narrowed, an allocation rule the paper does not explore.","The 3.3-point figure comes from three volunteers and a fixed budget of 30; generalizing it to larger populations and longer budgets would require a study the paper does not report.","In applications where the oracle is replaced by graded user attributes, the natural next formulation treats the dimension answers as noisy observations rather than exact values."],"forward_implications":["At a total budget of $B=30$ queries, ORACLE BO reports on average 3.3 points higher user satisfaction than the interpolated-audiogram baseline, meaning the gain is achievable with a modest home listening session.","On synthetic staircase functions, the best split spends roughly 15 of $B=90$ queries on dimension hints; spending too many or too few raises regret, so the allocation between query types is a tunable parameter.","Choosing dimension hints from coordinates where the satisfaction function varies most gives lower regret than random choices, so where the hints are taken from matters.","The benefit degrades gracefully as fewer dimension hints are available: with fewer hints, more whole-filter ratings are needed to reach the same satisfaction.","Because ORACLE BO is built on random-embedding sparsity methods, the hybrid-query idea is not tied to one embedding scheme; the same BAF/DMS design can ride other sparse Bayesian optimizers."],"supporting_citations":[{"why":"Supplies the ALEBO random-embedding framework that ORACLE BO modifies with batch acquisition and dimension-matched sampling.","marker":"Letham et al. 2020"},{"why":"Provides the Bayesian optimization and expected-improvement machinery on which BAF's batch metric is built.","marker":"Frazier 2018"},{"why":"Gives the Gaussian-process regression posterior derivation used to update the surrogate after each filter query.","marker":"Wang 2020"},{"why":"Supplies slice sampling, the MCMC method BAF uses to draw batches of candidate filters from the posterior.","marker":"Neal 2003"},{"why":"Provides the staircase benchmark functions used to simulate human satisfaction scoring.","marker":"Al-Roomi 2015"},{"why":"Provides the Branin, Hartmann6, and Rosenbrock benchmarks used to test ORACLE BO beyond staircase functions.","marker":"Sonja Surjanovic 2013"},{"why":"Grounds the audiogram interpretation used both as the dimension-query baseline and as the motivating application.","marker":"Salmon et al. 2022"},{"why":"Provides the NHANES hearing-loss profiles used to corrupt audio and emulate hearing-impaired listeners in the user study.","marker":"CDC 2011"}],"fun_headline_variants":["Hybrid queries boost audio personalization within budget","Ask users for both ratings and hints to tune audio","Combining filter ratings and element hints improves audio tuning","Hybrid feedback strategy beats single-query audio optimization","Pointwise user hints speed up audio personalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes an oracle that answers each requested coordinate $h^*[j]$ of the true optimal filter exactly, and in the motivating hearing-aid setting the paper substitutes coarse audiogram measurements for that oracle.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid queries boost audio personalization within budget","Ask users for both ratings and hints to tune audio","Combining filter ratings and element hints improves audio tuning","Hybrid feedback strategy beats single-query audio optimization","Pointwise user hints speed up audio personalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000312,"raw_usage":{"total_tokens":1810,"prompt_tokens":1016,"completion_tokens":794,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":720}},"tokens_in":632,"tokens_out":794,"duration_ms":7135,"temperature":1.0,"reasoning_tokens":720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:39:22.829168+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ORACLE BO on a synthetic staircase function while corrupting each dimension-query answer with independent noise (say, a few dB of error on the requested gain); if a few noisy hints eliminate the advantage over pure filter-query Bayesian optimization at the same budget, the claimed gain depends on exact oracle answers rather than on the hybrid structure itself.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ALEBO random-embedding framework that ORACLE BO modifies with batch acquisition and dimension-matched sampling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies slice sampling, the MCMC method BAF uses to draw batches of candidate filters from the posterior."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the staircase benchmark functions used to simulate human satisfaction scoring."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Branin, Hartmann6, and Rosenbrock benchmarks used to test ORACLE BO beyond staircase functions."},{"cited_title":"K.; Brant, J.; Hohman, M","cited_arxiv_id":null,"evidence_quote":"Grounds the audiogram interpretation used both as the dimension-query baseline and as the motivating application."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the NHANES hearing-loss profiles used to corrupt audio and emulate hearing-impaired listeners in the user study."}],"review_version":1}