REVIEW 4 major objections 4 minor 7 references
Mixed-variable policy-based optimization
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that policy-based optimization can be extended to mixed-variable problems by factorizing the policy into an independent multivariate normal for continuous variables and categorical distributions for discrete variables…
desk verdict A clean, incremental extension of PBO to mixed variables, but the single mirror experiment with no baseline or ablation leaves 'robustness' unsupported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the factorized mixed policy, a product of a multivariate normal distribution over continuous variables and several categorical distributions over discrete variables. The log-probability of a full action is the sum of the component log-probabilities, so the advantage estimate and loss computation from the original continuous policy-based optimization carry over unchanged. Around this factorization, the paper modifies the base method: the mean vector is updated by weighted recombination of elite points instead of being generated by a neural network, and a fixed number of elite points (typically half the population) is retained each generation. This combination is what lets a single-step policy-gradient algorithm handle discrete and continuous decisions simultaneously.
What would settle it
Run the algorithm on a mixed-variable problem whose known optimum requires different continuous ranges for different discrete choices; if the independent factorized policy consistently stops above the known optimum while a policy with a conditional discrete head reaches it, the central claim fails.
Extended reading notes
Core claim
The central claim is that the factorized policy $\pi_\theta(a) = \pi_{\theta_c}(a_c)\pi_{\theta_d}(a_d)$, with $\log\pi_\theta(a) = \log\pi_{\theta_c}(a_c) + \log\pi_{\theta_d}(a_d)$, converts mixed-variable optimization into the same single-step policy-gradient problem as the continuous-only version. A neural network outputs logits for all categorical choices at once; at sampling time those logits are split per discrete variable, and the continuous and discrete samples are drawn independently. The paper then shows on the 20-layer TiO2/MgF2 mirror problem that this recipe reaches high average reflectance in roughly 7,500 evaluations and, with a flatness penalty, produces a flatter spectrum with mean reflectance 0.907. The result is offered as evidence that the mixed-variable extension is natural, robust, and does not require inner/outer loops or surrogate models.
Load-bearing premise
The load-bearing premise is that independent sampling of material choices and thicknesses, combined as a product of probabilities, does not miss the couplings that determine good designs.
Editorial extensions
If this is right
- The single-step policy-gradient machinery for continuous optimization now applies, without new estimators, to problems that also contain discrete choices.
- On the 20-layer dielectric mirror, the method reaches an average reflectance of 0.9306 over 300 to 500 nm in about 7,500 evaluations.
- With a flatness penalty, the optimizer still finds good designs at mean reflectance 0.907, but with higher run-to-run variance.
- Since the categorical logits come from one network layer, adding discrete variables or categories translates into enlarging that output layer.
- The weighted-recombination mean update is introduced to stabilize updates that the neural-network mean generator could destabilize.
Reading between the lines
- The paper leaves untested whether a policy that makes the discrete choice conditional on the continuous sample would perform better; such a comparison would show whether the independence assumption is a real loss.
- The sum-of-log-probabilities construction is generic, so the same recipe could be applied to other single-step policy-gradient optimizers, not only the version presented here.
- The flatness-penalty experiment suggests reward shaping carries much of the burden; a multiobjective formulation treating mean reflectance and flatness separately would be a natural follow-up.
- Reporting only the dielectric-mirror problem means the method's robustness claim would be strengthened by a benchmark suite with known mixed-variable optima.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an extension of the policy-based optimization (PBO) method to mixed-variable problems. The algorithm independently samples continuous variables from a multivariate normal distribution and discrete variables from categorical distributions, with the joint policy log-probability computed as the sum of the continuous and discrete log-probabilities. Two modifications to the original PBO are introduced: a weighted recombination of elite points for the mean update and a fixed elite fraction. The method is tested on a 20-layer dielectric mirror design problem with 40 degrees of freedom (20 continuous thicknesses and 20 discrete materials), maximizing average reflectance over 300–500 nm. A second objective adds a flatness penalty. The paper reports an average reflectance of 0.9306 and states that convergence is reached in approximately 7.5k evaluations.
Significance. If validated, the proposed extension is a simple and potentially useful addition to the PBO family, applicable to black-box mixed-variable optimization. The formulation is natural and could be easily implemented in existing PBO codebases. The objective function (average reflectance via the transfer matrix method) is external to the algorithm, and no fitted parameter defines the target result, so the central claim is not circular. However, the evidence is limited to a single problem with five runs, no baselines, no ablations, and no hyperparameter details, so the claims of 'high-quality solutions' and 'robustness' are not quantitatively supported. The paper serves as a proof-of-concept rather than a fully validated method, and its significance is correspondingly modest.
major comments (4)
- [Section 4] The central claim of robustness is unsupported by the experimental evidence. The paper reports results on a single 20-layer dielectric mirror, with five runs, and provides no baseline comparison (e.g., random search, CMA-ES with mixed-variable handling, or the continuous-only PBO with a discrete encoding). The average reflectance of 0.9306 and the cost evolution curves in Figures 1 and 2 do not quantify the variability in the final objective value; 'moderate standard deviation' is not a numerical statement. To support the claims in the Abstract and Conclusion, the authors should add quantitative statistics (mean, standard deviation, min/max over runs) and compare against at least one baseline on the same problem, or clearly scope the claim to 'demonstration of feasibility'.
- [Section 3] The independent factorization of the joint policy into continuous and discrete policies is the key algorithmic novelty, but its adequacy is untested. The paper states 'Given the independence of the continuous πθc and discrete πθd policies' without justification or experimental validation. In the dielectric mirror problem, the discrete material choice of a layer changes the optimal thickness of neighboring layers through thin-film interference; a factorized policy cannot represent such coupling. The paper should provide an ablation, e.g., a coupled policy that conditions discrete decisions on continuous samples (or vice versa), or a toy problem where the continuous optimum shifts with the discrete choice, to determine whether the factorization is restrictive in practice. Without this, the generality claim 'handles mixed-variables problems in a natural way' is not established.
- [Section 2] The two modifications to the original PBO—the weighted-recombination mean update and the fixed elite fraction—are introduced without supporting evidence. In particular, the claim that the neural-network-generated mean 'could sometimes lead to unstable updates' is anecdotal, and no comparison between the updated and original PBO is shown. Since these changes are part of the proposed algorithm, their effect on performance should be quantified, for example by running the original PBO and the revised PBO on the same continuous benchmark. If the changes are intended as independent contributions, they need their own validation; if they are merely engineering choices, the paper should say so explicitly and explain why they do not affect the validity of the mixed-variable extension.
- [Section 4] The experimental setup is not reproducible. The paper does not report population size, number of generations, elite fraction (beyond 'usually half'), learning rates, network architectures, or the random seeds used for the five runs. It also omits the implementation details of the transfer matrix method (e.g., wavelength discretization). For a methods paper, these details are essential; without them, the 'robustness' claim cannot be independently verified. Please provide a table of hyperparameters and a description of the computational setup.
minor comments (4)
- [Introduction] There are several typos and grammatical errors: 'fundational' should be 'foundational', 'aiming a providing' should be 'aiming at providing', 'titanium dioxyde' should be 'titanium dioxide', 'expanse' should be 'expense', and 'reflctance' should be 'reflectance'.
- [Section 4] The figure captions (Figures 1 and 2) refer to 'Top right' and 'Bottom right' panels, but the captions do not explicitly label these as (a), (b), etc. Please add explicit panel labels for clarity.
- [Abstract / Keywords] The keywords line has inconsistent spacing: 'policy based optimization· mixed variable optimization' should use spaces around the middle dot, e.g., 'policy based optimization · mixed variable optimization'.
- [Section 4] The paper states 'Convergence is reached in approximately 7.5k evaluations' but the figures show 10k evaluations on the x-axis. Please clarify whether training was stopped at 7.5k or if this is an estimate, and describe the stopping criterion.
Circularity Check
No significant circularity; the mixed-variable PBO result is validated against an external transfer-matrix objective.
full rationale
The derivation chain is not circular. The objective is the average reflectance computed by the transfer matrix method (Section 4), a fully external physics simulator with fixed material indices (n_TiO2 = 2.4, n_MgF2 = 1.38) and fixed substrate/air indices; no algorithm parameter is fitted to define the reported reflectance values 0.9306 or 0.907. The mixed-variable extension in Section 3 is an explicit construction: a factored policy pi_theta(a) = pi_theta_c(a_c) * pi_theta_d(a_d) with log-probability sum; this is the algorithm's mechanism, not a prediction derived from itself. The use of the author's prior PBO [6] for the covariance update, whitened advantage, and loss is a citation of an external, previously published algorithm used as a component; the current paper's central empirical claim is tested against an independent benchmark (the 20-layer mirror), so the self-citation is not load-bearing in the sense of making the result true by definition. The revised mean update is presented as a modification for stability and is itself evaluated by the same external objective. No fitted parameter is renamed as a prediction, and no uniqueness theorem is invoked. The absence of baselines or coupled-policy ablations is a correctness and robustness concern, not circularity.
Assumptions & free parameters
free parameters (3)
- Flatness penalty weight alpha =
0.1
- Elite fraction =
0.5 (stated as 'usually equal to half')
- Training hyperparameters (population size, learning rates, network architecture, optimizer, seeds)
assumptions (3)
- domain assumption Transfer matrix method with fixed refractive indices gives an adequate objective for the dielectric mirror design (Section 4, 'The computation of the reflectance is performed using the transfer matrix method').
- domain assumption The policy-gradient update rules from [6] remain valid for the factorized mixed-variable policy with only the log-probability addition (Section 3, 'the computation of the advantage vector and the loss function remain similar to what is described in [6]').
- ad hoc to paper Whitened rewards used as recombination weights yield a stable mean update (Section 2, mean update formula).
Cite this review
Pith. "Pith review of Mixed-variable policy-based optimization." pith.science (2026). https://pith.science/paper/VRKFAHVE
@misc{pith2026250613240,
author = {Pith},
title = {Pith review of: Mixed-variable policy-based optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/VRKFAHVE}},
note = {Machine review of arXiv:2506.13240}
}
read the original abstract
The optimization of mixed-variable problems remains a significant challenge. We propose an extension of the policy-based optimization method that handles mixed-variables problems in a natural way, through a simple policy combination. This is achieved by independently sampling from a multivariate normal distribution for the continuous domain, and from multiple categorical distributions for the discrete choices. Results demonstrate that the agent successfully yields high-quality solutions on a classical problem of electromagnetics, showcasing its robustness.
Figures
Reference graph
Works this paper leans on
-
[6]
J. Viquerat, R. Duvigneau, P. Meliga, A. Kuhnle, and E. Hachem. Policy-based optimization: single-step policy gradient method seen as an evolution strategy. Neural Computing and Applications, 35:449–467, 2023
work page 2023
-
[1]
R. Fletcher and S. Leyffer. Solving mixed integer nonlinear programs by outer approximation. Mathematical Programming, 66(1-3):327–349, 1994
work page 1994
-
[2]
M. A. Duran and I. E. Grossmann. An outer-approximation algorithm for a class of mixed- integer nonlinear programs. Mathematical Programming, 36(3):307–339, 1986
work page 1986
-
[3]
C. A. Coello Coello. A comprehensive survey of evolutionary-based multiobjective optimiza- tion techniques. Knowledge and Information Systems, 1(3):269–308, 2002
work page 2002
-
[4]
E. C. Laskari, K. E. Parsopoulos, and M. N. Vrahatis. Particle swarm optimization for integer programming. In Proceedings of the 2002 IEEE Congress on Evolutionary Computation (CEC 2002), volume 2, pages 1582–1587, 2002
work page 2002
-
[5]
E. C. Garrido-Merch´ an and D. Hern´ andez-Lobato. Dealing with categorical and integer variables in bayesian optimization with gaussian processes. Neurocomputing, 380:20–35, 2020
work page 2020
-
[7]
Eugene Hecht. Optics. Pearson, Boston, 5th edition, 2017. 5
work page 2017
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.