{"id":"1475693e-90c4-4f88-9327-4c3972ceba93","arxiv_id":"2502.06585","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A modular 'Extract-QD' framework and a new Extract-ME algorithm that re-evaluates archive elites, consistently matching or outperforming previous uncertain-QD methods on standard benchmarks.","lead":"This paper introduces a modular framework for Quality-Diversity optimization under uncertainty, and a new method, Extract-ME, that re-evaluates archive elites to handle noisy evaluations. The new method matches or beats existing uncertain-QD methods on standard benchmarks, and the framework improves the PGA-MAP-Elites algorithm at no extra evaluation cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal inconsistency in EME's evaluation budget: Section 3.1.1/4.2.3 imply N=1, Table 1 says N=2; the 'no additional evaluation cost' claim is unverifiable.","rationale":"The central claim of the paper is empirical: EME matches or beats the best UQD method on standard benchmarks, and EQD-augmented PGA improves at no extra evaluation cost. Both claims depend on a precise evaluation budget. I found a concrete internal contradiction: Section 3.1.1 and Section 4.2.3 describe a 25% extraction budget with 96 new offspring per 128 evaluations, which only works if each offspring receives N=1 sample, but Table 1 (with the definition in Section 3.2.1) specifies N=2 for EME. This is not a cosmetic typo: with N=2, the described evaluation counts exceed the budget by 75%, so either the algorithm, the budget accounting, or the reported comparisons are mis-specified. The reader's concern about fixed hyperparameters (25%, d=8, rank-based extraction, N=2) is real and remains an important limitation, but it presumes the algorithm as described is the algorithm run. The evaluation-accounting inconsistency is more fundamental because it blocks even a consistent re-implementation from the text. The lack of released code (Section 4.1.2) makes this unverifiable. I therefore recommend keeping the conditional verdict but explicitly requiring the authors to clarify EME's per-generation evaluation counts and to release code; hyperparameter sensitivity should be added as a further condition.","tokens_in":17043,"tokens_out":8942,"duration_ms":70211,"concrete_test":"Implement EME from the paper's description and Table 1 (N=2, 25% extraction budget) on the Ant task with sampling-size 1024, and count total evaluations per generation. If 25% extraction implies 256 re-evaluations and 768 evaluations for new offspring, then with N=2 only 384 new offspring are evaluated; Section 4.1.5's convergence and QD-scores must be re-checked against this count. Alternatively, if the released code shows N=1, then Table 1's N=2 is an error. This single accounting check determines whether the 'no additional evaluation cost' claim in Section 4.2 is valid.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing concern is an internal inconsistency in EME's evaluation accounting. Section 3.1.1 defines EME as dedicating 25% of the per-generation evaluation budget to re-evaluating extracted elites, with the example: 'if ME evaluates 128 offspring per generation, EME would only generate and evaluate 96 new offspring, and use the remaining 32 evaluations to re-evaluate elites'. Section 4.2.3 repeats this for EPGA at 128 evaluations per generation. However, Table 1 lists EME's 'Samples' parameter as N=2 (the number of samples spent on the first evaluation of solutions, per Section 3.2.1). With N=2, 96 new offspring require 192 evaluations, not 96, so the total would be 224, exceeding the stated budget of 128. If N=1 is intended, then Table 1 and the definition of the Samples module are wrong. This ambiguity directly affects the central claim that EPGA improves PGA 'at no additional evaluation cost' (Section 4.2.4) and the correctness of EME's comparison under the fixed sampling-size protocol of Section 4.1.4. Since the code is not released ('URL-to-be-released-upon-acceptance', Section 4.1.2), the reader cannot determine which specification was actually implemented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Extract-QD (EQD) Framework, a modular decomposition of uncertain quality-diversity (UQD) algorithms, and Extract-ME (EME), an instantiation built on MAP-Elites that re-evaluates a fraction of archived elites each generation using a depth-based archive and per-solution evaluation buffers. EME is evaluated on four UQD benchmarks against Vanilla-ME, ME-Sampling, Adapt-ME, AS, and Deep-Grid, reporting corrected QD-score and average samples. A second experiment instantiates the framework on PGA-MAP-Elites (EPGA) and reports improved corrected QD-score on three QD-RL tasks at the same nominal evaluation budget. The paper claims that EME consistently matches or outperforms the best compared method and that the framework can be used to augment existing QD algorithms at no additional evaluation cost.","tokens_in":17347,"tokens_out":9302,"duration_ms":75945,"significance":"If the empirical claims hold, the paper would provide a useful unifying taxonomy and a strong default UQD algorithm. The use of a corrected QD-score, ten seeds, and Holm-Bonferroni-corrected Wilcoxon tests is a sound evaluation methodology, and the modular framework is a helpful organizational contribution for practitioners. The EPGA result, if the evaluation-budget accounting is correct, is a practical improvement at no additional sampling cost. However, the significance is tempered by missing sensitivity analysis and by the budget-accounting ambiguity that must be resolved before the headline comparisons can be trusted.","major_comments":[{"comment":"Section 3.1.1 and Section 4.2.3 define EME/EPGA's per-generation budget as 25% extraction and 75% new offspring, with the example that 128 evaluations per generation means 96 new offspring and 32 elite re-evaluations. Table 1, however, lists EME's Samples parameter as N=2, which Section 3.2.1 defines as the number of samples spent on the first evaluation of solutions. Under the sampling-size protocol defined in Section 4.1.4, where ME-Sampling with N=32 is said to consume 32 evaluations per offspring, 96 new offspring with N=2 consume 192 evaluations; together with 32 re-evaluations the total is 224, not 128. If N=1 was used in the experiments, Table 1 and the Samples module are mis-specified; if N=2 was used, the 'no additional evaluation cost' claim in Section 4.2.4 and the fair-budget comparison are not supported. Since the code is only a placeholder URL (Section 4.1.2), this ambiguity cannot be resolved from the manuscript.","section":"3.1.1, 4.2.3, Table 1"},{"comment":"The extraction probability is described as 'exponentially-proportional to their rank in their cell', but no formula, normalization, or parameter is given. This is an essential component of EME and of any reproduction of the experiments; without code or a precise definition, the method is underspecified. Please provide the exact distribution (or pseudo-code) and, ideally, a sensitivity check on its shape.","section":"3.1.3"},{"comment":"EME's main hyperparameters are fixed without sensitivity analysis: 25% extraction proportion, depth d=8, exponential rank-based extraction, and N=2. Since the paper's abstract recommends EME as a reliable 'first guess' for any new uncertain task, the authors should either report sensitivity experiments over plausible ranges of these parameters or moderate the generality claim to the tested configuration.","section":"3.1.1, 3.1.3, Table 1"},{"comment":"The phrase 'consistently performs at least as well as the best-performing method for each task' is stronger than the evidence: AS is not defined on Hexapod and Ant (Section 4.1.5) and is therefore absent from exactly half of the benchmark tasks, yet AS is one of the strongest methods where it runs. Please either phrase the claim as 'best-performing method among those defined on the task', or provide a feasible AS variant for large archives to make the comparison complete.","section":"4.1.5"}],"minor_comments":[{"comment":"Section 2.2.2 cites ME-Sampling as [9], while Section 3.2.2 and Table 1 cite it as [28]; reference [28] is also used for Adapt-ME in Section 2.2.3. The correct source should be identified consistently.","section":"2.2.2, 3.2.2, Table 1"},{"comment":"Section 4.1.2 states that code will be released at 'URL-to-be-released-upon-acceptance'; for a paper whose central contribution is empirical and whose algorithm is underspecified in places, providing code or at least a detailed pseudo-code appendix would considerably strengthen reproducibility.","section":"4.1.2"},{"comment":"The symbols p_b and min(p_b, C) are not defined in the table or in the surrounding text; please define them explicitly.","section":"Table 1"},{"comment":"The paper reports 'p-values using the Wilcoxon test with Holm-Bonferroni correction' but does not state the number of tests or the family over which the correction is applied; please clarify.","section":"4.1.3"},{"comment":"There are several small grammatical issues, for example 'In this section, we first introduced its modules' in Section 3.2.1 and 'MOME-X build on top of existing multi-objective QD works' in Appendix A; these should be corrected.","section":"3.2.1, Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The paper's contribution is incremental but potentially useful, and the experimental methodology is generally solid. The main risk is the budget-accounting ambiguity rather than any suspicion of misconduct; the self-citation pattern is heavy but the cited methods are directly relevant and attributed. I would recommend asking for a clear audit of the evaluation-budget accounting and, if possible, a release of code or detailed pseudo-code in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the stress-test note is correct: there is an internal inconsistency in EME's evaluation accounting. Section 3.1.1's example assumes each new offspring costs one evaluation (96 offspring + 32 re-evals = 128), but Table 1 sets N=2 for EME's first evaluation samples, which would double the offspring cost to 192 and push the total to 224. Since the paper compares methods under a fixed per-generation sampling budget, this ambiguity directly affects the 'no additional evaluation cost' claim and the fairness of the EME comparison. It's likely a simple fix — clarify whether N=1 or adjust the example — but as written it is a real flaw.\n\nWhat the paper does well: the EQD framework is a genuinely useful modular decomposition of existing UQD methods, and the table mapping methods to modules is a handy reference. EME itself is a clean adaptive-sampling variant: it fixes the per-generation extraction budget (unlike AS, which re-evaluates the whole archive) and uses depth plus rank-based extraction. Empirically, EME performs well across the four benchmark tasks, and the EPGA variant shows that plugging extraction into PGA-ME improves performance at the same evaluation budget. The experimental methodology — corrected QD-score, 10 seeds, Holm-Bonferroni-corrected p-values — is solid.\n\nThe soft spots, in order of seriousness: (1) the budget inconsistency above; (2) no code (the URL is a placeholder), which is frustrating in a paper whose main claims are empirical; (3) hyperparameters are fixed with no sensitivity analysis — the 25% extraction proportion, depth 8, exponential rank probability, and N=2 are all choices that could affect the 'first guess' generalization claim; (4) the 'consistent first guess' claim is extrapolated from four tasks, two of which are low-dimensional Arm variants; (5) AS is undefined on two of the four tasks, so the strongest adaptive-sampling baseline is absent there. None of these are fatal, but together they make the central empirical claim shakier than the text admits.\n\nWho gets value: anyone working on QD under uncertainty or wanting to add uncertainty handling to existing QD algorithms. The framework is worth having even if EME is incremental.\n\nMy recommendation: send it to peer review with a major-revision request. The framework and EME deserve a serious referee, but the budget ambiguity must be resolved and the code should be released before acceptance.","headline":"Useful modular framework and a solid but incremental EME variant, undermined by an internal inconsistency in the evaluation budget and missing code.","tokens_in":17854,"tokens_out":2505,"would_cite":true,"duration_ms":21339,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Extract-ME, a new adaptive-sampling method, is claimed to match or beat the best existing uncertain-quality-diversity method on every standard benchmark, and the Extract-QD Framework lets the same mechanism be bolted onto algorithms such…","keywords":["Quality-Diversity Optimisation","Uncertain Domains","MAP-Elites","Behavioural Diversity","Noisy Optimisation","Adaptive Sampling","Extract-ME","PGA-MAP-Elites"],"falsifier":"Run EME on the same four task suites while sweeping the extraction budget over 5%, 25%, and 50% and the depth over 1, 2, 8, and 32, scoring with the paper's Corrected QD-Score. If a non-default setting clearly wins on several tasks, or if EME with its fixed settings falls clearly below the best baseline on any of the four tasks, the 'first guess' and consistency claims are falsified.","tokens_in":16869,"feed_emoji":"🎯","tokens_out":9543,"duration_ms":151325,"temperature":0.7,"pith_summary":"Quality-Diversity search usually assumes each solution is evaluated once, but in noisy, stochastic, or uncertain domains a 'lucky' evaluation can fill the archive with illusions. The paper's aim is to make Uncertain Quality-Diversity (UQD) easy to adopt: it proposes a modular Framework that re-describes every existing UQD method as a combination of interchangeable parts, and it instantiates the framework as Extract-ME (EME), an adaptive-sampling algorithm. EME spends a fixed 25% of each generation's evaluations re-evaluating elites drawn from a depth-$d$ archive, with rank-weighted extraction probability, and merges those re-evaluations with offspring through the same insertion rule. The paper reports that EME matches the best specialist method on every benchmark it runs and beats all baselines on the Ant task, whereas each existing method fails somewhere. The main experiments deliberately address only the performance-estimation sub-problem; reproducibility-aware variants are represented in the framework but not empirically tested here.","feed_headline":"New method beats or ties every rival on noisy quality-diversity tasks","feed_subtitle":"Re-evaluating a fixed share of archive elites each generation lifts uncertain QD performance with no extra evaluations.","key_machinery":"Three components carry the argument. The extraction operator removes a randomly chosen, rank-weighted subset of elites from the archive each generation and budgets their re-evaluations alongside offspring; the depth-augmented container keeps $d=8$ promising candidates per cell so that a re-evaluated solution whose descriptor 'drifts' can be replaced without emptying the cell; and per-solution evaluation buffers accumulate fitness and descriptor samples so every re-check improves the estimate. Taken together they convert a fixed per-generation evaluation budget into a balance between exploration (new offspring) and exploitation of reliable estimates (elite re-evaluation). The EQD Framework's contribution is to expose each of these as swappable modules, so EME is just one point in a design space that also contains ME-Sampling, Deep-Grid, Archive-Sampling, Adapt-ME, and the reproducibility-aware variants.","core_discovery":"The central claim is that a single extraction mechanism, applied generically, removes the need to choose among UQD methods: re-evaluate a fixed share of the archive each generation, keep several candidate elites per cell, and let every solution accumulate its evaluations in a buffer. EME is this mechanism instantiated with 25% extraction, depth $d=8$, exponential rank-based extraction probability, and $N=2$ first-evaluation samples. The paper states that EME 'consistently performs at least as well as the best-performing method for each task' across Arm, Hexapod, Walker, and Ant benchmarks, and that the same budget split, when applied to PGA-MAP-Elites via the framework, yields Extract-PGA with higher Corrected QD-Score on all three QD-RL tasks at no additional evaluation cost. It further claims that the Framework's six modules—selection, variation, extraction, container with depth, depth-ordering, and sample count—encompass all surveyed prior approaches as instantiations, so practitioners can derive task-specific methods by swapping modules rather than designing from scratch.","pith_inferences":["If the framework's unification claim is right, then reporting future UQD results as a tuple of six module choices would let papers be compared by construction, without re-running baselines; the paper does not itself propose this reporting convention.","The fixed 25% extraction budget and depth 8 are never swept, so a natural next experiment is to make the extraction fraction depend on noise level or archive size; the paper gives no evidence on how sensitive EME is to these settings.","EME's advantage over Archive-Sampling is concentrated on top-ranked elites, which suggests it should matter most when descriptor noise is large relative to cell size or when full-archive re-evaluation is infeasible; the Ant result points that way, but the paper does not state this as a prediction."],"forward_implications":["A practitioner facing a new uncertain QD task can start from EME's fixed settings instead of comparing a menu of UQD methods.","Because extraction reuses the existing evaluation budget, uncertainty handling does not necessarily cost extra samples; the paper reports this for Extract-PGA relative to PGA-MAP-Elites.","Swapping modules—especially the depth-ordering operator—gives a path to inject reproducibility or user preference into any UQD method without redesigning the search.","Existing QD-RL algorithms can be made uncertainty-aware by wrapping their variation operator in the EQD loop, improving corrected archive quality on Walker, Cheetah, and Ant at equal evaluation count."],"supporting_citations":[{"why":"Defines MAP-Elites, the base archive algorithm that EME and all compared methods modify.","marker":"[34]"},{"why":"Supplies the modular QD Framework that the Extract-QD Framework extends.","marker":"[8]"},{"why":"Defines the Uncertain-QD problem, the corrected-archive evaluation methodology, and the Archive-Sampling baseline.","marker":"[14]"},{"why":"Introduces Adaptive-Sampling MAP-Elites, whose depth-of-archive idea EME adopts and whose sequential constraint EME removes.","marker":"[28]"},{"why":"Proposes Deep-Grid, the strong baseline on descriptor-noise tasks that EME must match.","marker":"[13]"},{"why":"Provides PGA-MAP-Elites, whose variation operator is swapped in to build Extract-PGA.","marker":"[35]"},{"why":"Supplies the reproducibility-weighted baselines and the benchmark tasks used in the main comparison.","marker":"[15]"},{"why":"Contributes ME-Low-Spread and the QD-RL task settings used in the second experiment.","marker":"[33]"}],"fun_headline_variants":["Generic framework unifies noisy QD, new method wins or ties","One extraction rule beats all noisy quality-diversity rivals","No-cost upgrade for uncertain QD: re-evaluate 25% of archive","EQD: Swap modules to build task-specific QD methods","First-guess method EME matches best uncertain QD results"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper fixes EME's hyperparameters—25% extraction budget, depth 8, exponential rank-based extraction, and two first-evaluation samples—without a sensitivity study, so the 'first guess' claim assumes these hand-picked values generalize beyond the four benchmark tasks tested.","fun_headline_variants_meta":{"raw":{"variants":["Generic framework unifies noisy QD, new method wins or ties","One extraction rule beats all noisy quality-diversity rivals","No-cost upgrade for uncertain QD: re-evaluate 25% of archive","EQD: Swap modules to build task-specific QD methods","First-guess method EME matches best uncertain QD results"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1589,"prompt_tokens":1023,"completion_tokens":566,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":477}},"tokens_in":639,"tokens_out":566,"duration_ms":5357,"temperature":1.0,"reasoning_tokens":477,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T14:56:38.654582+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run EME on the same four task suites while sweeping the extraction budget over 5%, 25%, and 50% and the depth over 1, 2, 8, and 32, scoring with the paper's Corrected QD-Score. If a non-default setting clearly wins on several tasks, or if EME with its fixed settings falls clearly below the best baseline on any of the four tasks, the 'first guess' and consistency claims are falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Uncertain-QD problem, the corrected-archive evaluation methodology, and the Archive-Sampling baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Adaptive-Sampling MAP-Elites, whose depth-of-archive idea EME adopts and whose sequential constraint EME removes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Proposes Deep-Grid, the strong baseline on descriptor-noise tasks that EME must match."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides PGA-MAP-Elites, whose variation operator is swapped in to build Extract-PGA."},{"cited_title":"Exploring the Performance-Reproducibility Trade-off in Quality-Diversity","cited_arxiv_id":"2409.13315","evidence_quote":"Supplies the reproducibility-weighted baselines and the benchmark tasks used in the main comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes ME-Low-Spread and the QD-RL task settings used in the second experiment."}],"review_version":1}