{"id":"df1a8a60-2955-480e-b0f0-fbaea1e1b681","arxiv_id":"2506.13240","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A policy-based optimizer is extended to mixed continuous and categorical variables through independent Gaussian and categorical policies, and demonstrated on a 20-layer dielectric mirror with 40 degrees of freedom.","lead":"An optimization method based on reinforcement learning is extended to handle problems with both continuous and discrete variables by sampling each type from its own probability distribution. It is demonstrated on a 20-layer dielectric mirror, reaching an average reflectance of 0.93, but with no comparison to other optimizers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The independent factorization of continuous and discrete policies is untested; the single mirror example cannot establish 'high-quality' or 'robustness' without a coupled-policy ablation or baseline comparison, so the central claim remains conditional.","rationale":"The reader's weakest assumption identifies the independence of the continuous and discrete policies as the load-bearing premise, and I agree. The paper's evidence is a single dielectric mirror problem with no ablation, baseline, or hyperparameter disclosure, so the conditional verdict is appropriate. My stress-test does not reveal an internal mathematical error in the PBO extension; the policy-gradient combination of Gaussian and categorical log-probabilities is coherent. The most serious gap is empirical: the reported results cannot show that the factorized policy is a generally working mixed-variable optimizer, because the problem may not exercise coupling and no comparison is made. The concrete test I propose would directly settle whether the factorization is a real limitation. Since this concern matches the reader's conditionality rather than changing it, I recommend the verdict remain CONDITIONAL.","tokens_in":3622,"tokens_out":6121,"duration_ms":75479,"concrete_test":"Construct a synthetic mixed-variable benchmark where the continuous optimum depends on the discrete choice, e.g., f(x,z)=Σ_i (x_i - a_{z_i})^2 with a_0=0, a_1=5, plus a multimodal coupling term. Run the proposed factorized PBO and a coupled variant whose categorical logits are conditioned on the current continuous sample, with identical budgets and 20 seeds. If the factorized version's final value is more than 10% worse or requires more than 2x evaluations to match the coupled variant, the independence assumption is not benign and the central claim is overbroad.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the Section 3 assumption that the joint policy factorizes as πθc·πθd, with independent categorical distributions over layers. This assumption is never tested against a coupled policy or a problem with strong discrete-continuous interaction. The 20-layer mirror has 2^20 material combinations, but only ~7.5k evaluations per run are performed, and thin-film interference makes the optimal thickness of a layer depend on the material choices of neighboring layers. A factorized policy cannot represent such coupling, so even the reported 0.9306 average reflectance does not separate the method's contribution from the ease of the specific problem. No baseline, random search, or coupled-policy comparison is provided, and no hyperparameters are given, so the robustness claim is quantitatively unsupported. This is a correctness risk for the general claim, not an internal inconsistency: the algorithm as described may work on this mirror while failing on mixed-variable problems where the discrete choice changes the continuous optimum.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an extension of the policy-based optimization (PBO) method to mixed-variable problems. The algorithm independently samples continuous variables from a multivariate normal distribution and discrete variables from categorical distributions, with the joint policy log-probability computed as the sum of the continuous and discrete log-probabilities. Two modifications to the original PBO are introduced: a weighted recombination of elite points for the mean update and a fixed elite fraction. The method is tested on a 20-layer dielectric mirror design problem with 40 degrees of freedom (20 continuous thicknesses and 20 discrete materials), maximizing average reflectance over 300–500 nm. A second objective adds a flatness penalty. The paper reports an average reflectance of 0.9306 and states that convergence is reached in approximately 7.5k evaluations.","tokens_in":3769,"tokens_out":4493,"duration_ms":45296,"significance":"If validated, the proposed extension is a simple and potentially useful addition to the PBO family, applicable to black-box mixed-variable optimization. The formulation is natural and could be easily implemented in existing PBO codebases. The objective function (average reflectance via the transfer matrix method) is external to the algorithm, and no fitted parameter defines the target result, so the central claim is not circular. However, the evidence is limited to a single problem with five runs, no baselines, no ablations, and no hyperparameter details, so the claims of 'high-quality solutions' and 'robustness' are not quantitatively supported. The paper serves as a proof-of-concept rather than a fully validated method, and its significance is correspondingly modest.","major_comments":[{"comment":"The central claim of robustness is unsupported by the experimental evidence. The paper reports results on a single 20-layer dielectric mirror, with five runs, and provides no baseline comparison (e.g., random search, CMA-ES with mixed-variable handling, or the continuous-only PBO with a discrete encoding). The average reflectance of 0.9306 and the cost evolution curves in Figures 1 and 2 do not quantify the variability in the final objective value; 'moderate standard deviation' is not a numerical statement. To support the claims in the Abstract and Conclusion, the authors should add quantitative statistics (mean, standard deviation, min/max over runs) and compare against at least one baseline on the same problem, or clearly scope the claim to 'demonstration of feasibility'.","section":"Section 4"},{"comment":"The independent factorization of the joint policy into continuous and discrete policies is the key algorithmic novelty, but its adequacy is untested. The paper states 'Given the independence of the continuous πθc and discrete πθd policies' without justification or experimental validation. In the dielectric mirror problem, the discrete material choice of a layer changes the optimal thickness of neighboring layers through thin-film interference; a factorized policy cannot represent such coupling. The paper should provide an ablation, e.g., a coupled policy that conditions discrete decisions on continuous samples (or vice versa), or a toy problem where the continuous optimum shifts with the discrete choice, to determine whether the factorization is restrictive in practice. Without this, the generality claim 'handles mixed-variables problems in a natural way' is not established.","section":"Section 3"},{"comment":"The two modifications to the original PBO—the weighted-recombination mean update and the fixed elite fraction—are introduced without supporting evidence. In particular, the claim that the neural-network-generated mean 'could sometimes lead to unstable updates' is anecdotal, and no comparison between the updated and original PBO is shown. Since these changes are part of the proposed algorithm, their effect on performance should be quantified, for example by running the original PBO and the revised PBO on the same continuous benchmark. If the changes are intended as independent contributions, they need their own validation; if they are merely engineering choices, the paper should say so explicitly and explain why they do not affect the validity of the mixed-variable extension.","section":"Section 2"},{"comment":"The experimental setup is not reproducible. The paper does not report population size, number of generations, elite fraction (beyond 'usually half'), learning rates, network architectures, or the random seeds used for the five runs. It also omits the implementation details of the transfer matrix method (e.g., wavelength discretization). For a methods paper, these details are essential; without them, the 'robustness' claim cannot be independently verified. Please provide a table of hyperparameters and a description of the computational setup.","section":"Section 4"}],"minor_comments":[{"comment":"There are several typos and grammatical errors: 'fundational' should be 'foundational', 'aiming a providing' should be 'aiming at providing', 'titanium dioxyde' should be 'titanium dioxide', 'expanse' should be 'expense', and 'reflctance' should be 'reflectance'.","section":"Introduction"},{"comment":"The figure captions (Figures 1 and 2) refer to 'Top right' and 'Bottom right' panels, but the captions do not explicitly label these as (a), (b), etc. Please add explicit panel labels for clarity.","section":"Section 4"},{"comment":"The keywords line has inconsistent spacing: 'policy based optimization· mixed variable optimization' should use spaces around the middle dot, e.g., 'policy based optimization · mixed variable optimization'.","section":"Abstract / Keywords"},{"comment":"The paper states 'Convergence is reached in approximately 7.5k evaluations' but the figures show 10k evaluations on the x-axis. Please clarify whether training was stopped at 7.5k or if this is an estimate, and describe the stopping criterion.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is very short and the experimental section is minimal. The proposed algorithm is straightforward and the idea has merit, but the claims outpace the evidence. I would encourage the editor to ask for a substantial revision that either adds the missing comparisons and ablations or significantly tempers the claims. The paper may also need to be repositioned as a technical note rather than a full research article, depending on the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe short version: this is a modest but real extension of policy-based optimization (PBO) to mixed continuous/discrete variables. The mirror experiment suggests the method works, but the evidence is too thin to support the 'robustness' claim in the abstract.\n\nWhat's new: The author replaces PBO's neural-network mean update with a weighted recombination of elites and adds a separate network that outputs logits for categorical variables. The joint policy factorizes into a Gaussian over continuous variables and a product of categoricals over discrete variables, and the log-probabilities are summed. This is a standard factorization, and the paper honestly calls the extension 'straightforward.' The original PBO [6] is continuous-only, so this is a genuine, if incremental, contribution. The mirror problem is a nice demonstration: the algorithm finds a 20-layer stack with average reflectance 0.93 and appears stable across five runs. The transfer-matrix method is an external, standard objective, so there is no circularity.\n\nSoft spots: The biggest issue is the absence of any baseline. No random search, no evolutionary algorithm, no Bayesian optimization, not even the original PBO on a continuous relaxation. Without that, 'high-quality' and 'robust' are unquantified. The independence assumption in Section 3 is also untested: for the mirror, the optimal thickness of a layer depends on the material choices of its neighbors, so a factorized policy might miss coupling. The paper should include an ablation against a coupled policy or a problem with strong continuous-discrete interaction. Also, no hyperparameters (population size, learning rates, elite fraction, seeds) or code are given, so the results are not reproducible. The flatness penalty case increases variance, which the paper attributes to local minima; plausible, but not investigated.\n\nThe math is sound and the citation pattern is fine, with [6] properly credited as the direct source. The paper is clearly written. But as a methods paper, it is not yet complete.\n\nWho is this for? Someone considering PBO for mixed-variable design problems might find this a useful starting point. But the evaluation is not at a level that would convince a careful reader. I'd send it back for major revision: add baselines, an ablation of the factorization, and code or detailed hyperparameters. If that is done, it could become a solid short paper. As is, I would not send it to peer review; the evidence is too thin for the claims, though the idea itself is not flawed.","headline":"A clean, incremental extension of PBO to mixed variables, but the single mirror experiment with no baseline or ablation leaves 'robustness' unsupported.","tokens_in":4306,"tokens_out":3658,"would_cite":false,"duration_ms":38281,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C56","90C59"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that policy-based optimization can be extended to mixed-variable problems by factorizing the policy into an independent multivariate normal for continuous variables and categorical distributions for discrete variables…","keywords":["policy-based optimization","mixed-variable optimization","black-box optimization","policy gradient","categorical distribution","multivariate normal distribution","dielectric mirror","evolution strategy"],"falsifier":"Run the algorithm on a mixed-variable problem whose known optimum requires different continuous ranges for different discrete choices; if the independent factorized policy consistently stops above the known optimum while a policy with a conditional discrete head reaches it, the central claim fails.","tokens_in":1375,"feed_emoji":"🪞","tokens_out":2752,"duration_ms":88579,"temperature":0.7,"pith_summary":"Mixed-variable optimization is hard because discrete choices make the search space combinatorial while the continuous variables remain non-convex. The paper proposes to make the optimizer's sampling policy itself mixed: continuous variables are drawn from a multivariate normal distribution, each discrete variable is drawn from its own categorical distribution, and the joint policy is the sum of the individual log-probabilities, leaving the policy-gradient update unchanged. On a dielectric mirror with 20 layers and 40 degrees of freedom, choosing between TiO2 and MgF2 and layer thicknesses between 50 and 150 nm, the method reaches an average reflectance of 0.9306 over 300 to 500 nm in about 7,500 evaluations. Adding a flatness penalty lowers the mean reflectance to 0.907 while reducing dips, at the cost of higher run-to-run variance. The paper's claim is that this simple independent factorization is enough to handle mixed-variable black-box problems naturally and robustly.","feed_headline":"20-layer mirror designed in 7.5k evaluations","feed_subtitle":"Independent Gaussian and categorical sampling yields 0.93 average reflectance over 300-500 nm.","key_machinery":"The load-bearing object is the factorized mixed policy, a product of a multivariate normal distribution over continuous variables and several categorical distributions over discrete variables. The log-probability of a full action is the sum of the component log-probabilities, so the advantage estimate and loss computation from the original continuous policy-based optimization carry over unchanged. Around this factorization, the paper modifies the base method: the mean vector is updated by weighted recombination of elite points instead of being generated by a neural network, and a fixed number of elite points (typically half the population) is retained each generation. This combination is what lets a single-step policy-gradient algorithm handle discrete and continuous decisions simultaneously.","core_discovery":"The central claim is that the factorized policy $\\pi_\\theta(a) = \\pi_{\\theta_c}(a_c)\\pi_{\\theta_d}(a_d)$, with $\\log\\pi_\\theta(a) = \\log\\pi_{\\theta_c}(a_c) + \\log\\pi_{\\theta_d}(a_d)$, converts mixed-variable optimization into the same single-step policy-gradient problem as the continuous-only version. A neural network outputs logits for all categorical choices at once; at sampling time those logits are split per discrete variable, and the continuous and discrete samples are drawn independently. The paper then shows on the 20-layer TiO2/MgF2 mirror problem that this recipe reaches high average reflectance in roughly 7,500 evaluations and, with a flatness penalty, produces a flatter spectrum with mean reflectance 0.907. The result is offered as evidence that the mixed-variable extension is natural, robust, and does not require inner/outer loops or surrogate models.","pith_inferences":["The paper leaves untested whether a policy that makes the discrete choice conditional on the continuous sample would perform better; such a comparison would show whether the independence assumption is a real loss.","The sum-of-log-probabilities construction is generic, so the same recipe could be applied to other single-step policy-gradient optimizers, not only the version presented here.","The flatness-penalty experiment suggests reward shaping carries much of the burden; a multiobjective formulation treating mean reflectance and flatness separately would be a natural follow-up.","Reporting only the dielectric-mirror problem means the method's robustness claim would be strengthened by a benchmark suite with known mixed-variable optima."],"forward_implications":["The single-step policy-gradient machinery for continuous optimization now applies, without new estimators, to problems that also contain discrete choices.","On the 20-layer dielectric mirror, the method reaches an average reflectance of 0.9306 over 300 to 500 nm in about 7,500 evaluations.","With a flatness penalty, the optimizer still finds good designs at mean reflectance 0.907, but with higher run-to-run variance.","Since the categorical logits come from one network layer, adding discrete variables or categories translates into enlarging that output layer.","The weighted-recombination mean update is introduced to stabilize updates that the neural-network mean generator could destabilize."],"supporting_citations":[{"why":"It defines the single-step policy-gradient method that this paper extends, supplying the advantage, whitened-reward, and covariance update that the mixed-variable version reuses.","marker":"[6]"},{"why":"It provides the transfer-matrix method used to compute the reflectance objective for the dielectric-mirror test case.","marker":"[7]"},{"why":"It represents the surrogate-based Bayesian optimization route for mixed categorical and integer variables, the main alternative against which the simplicity of the present extension is positioned.","marker":"[5]"}],"fun_headline_variants":["20-layer mirror in 7.5k evaluations with factorized policy","Independent Gaussian and categorical sampling optimize mirrors","Mixed-variable policy gradient: no inner loop, no surrogate","Natural mixed-variable optimization via factorized policy sampling"],"cache_read_input_tokens":6528,"weakest_assumption_plain":"The load-bearing premise is that independent sampling of material choices and thicknesses, combined as a product of probabilities, does not miss the couplings that determine good designs.","fun_headline_variants_meta":{"raw":{"variants":["20-layer mirror in 7.5k evaluations with factorized policy","Independent Gaussian and categorical sampling optimize mirrors","Mixed-variable policy gradient: no inner loop, no surrogate","Natural mixed-variable optimization via factorized policy sampling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000462,"raw_usage":{"total_tokens":2230,"prompt_tokens":786,"completion_tokens":1444,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":402,"completion_tokens_details":{"reasoning_tokens":1378}},"tokens_in":402,"tokens_out":1444,"duration_ms":12320,"temperature":1.0,"reasoning_tokens":1378,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:36:29.013157+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the algorithm on a mixed-variable problem whose known optimum requires different continuous ranges for different discrete choices; if the independent factorized policy consistently stops above the known optimum while a policy with a conditional discrete head reaches it, the central claim fails.","supporting_citations":[{"cited_title":"Viquerat, R","cited_arxiv_id":null,"evidence_quote":"It defines the single-step policy-gradient method that this paper extends, supplying the advantage, whitened-reward, and covariance update that the mixed-variable version reuses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the transfer-matrix method used to compute the reflectance objective for the dielectric-mirror test case."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It represents the surrogate-based Bayesian optimization route for mixed categorical and integer variables, the main alternative against which the simplicity of the present extension is positioned."}],"review_version":1}