REVIEW 3 major objections 4 minor 24 references
Optimization with Equality and Inequality Constraints Using Parameter Continuation
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Staged continuation reaches constrained optima from infeasible starts
desk verdict A genuine extension of the Kernévez–Doedel successive continuation framework to inequality constraints, with honest numerical demonstrations and a load-bearing reachability assumption that the paper itself concedes. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine of the method is the augmented system $F_{\mathrm{aug}}=0$ built from (i) the equality constraints $\Phi(u)=0$ and objective monitor $\Psi(u)-\mu=0$, (ii) the adjoint condition $(D\Phi(u))^*\lambda+(D\Psi(u))^*\eta+(DG(u))^*\sigma=0$ with normalization $\eta-\nu=0$, and (iii) the relaxed complementary conditions $\chi(\sigma_i,-G_i(u))-\kappa_i=0$ for each inequality. The complementarity function $\chi(a,b)=\sqrt{a^2+b^2}-a-b$ vanishes exactly when $a,b\ge 0$ and $ab=0$, so the equations enforce complementary slackness in the limit $\kappa=0$; its nonsmooth zero contour at the origin is what creates the singular points where branch switching to nonzero multipliers can occur. The continuation parameters $\nu$ and $\kappa$ are frozen or released stage by stage, and the lemmas show when the corresponding solution sets are one-dimensional manifolds, when a stationary point of the objective is a branch point, and when a secondary branch proceeds to the desired terminal values.
What would settle it
A concrete check: take a smooth constrained problem with a unique KKT point and try every admissible initial point and index order; if every staged path either stalls at a negatively aligned singular point or leaves the computational domain before $\nu=1$ and $\kappa=0$ are reached, the claimed search-from-infeasible-start property fails. The paper's own motivating example shows such a stall for one initial choice ($(3,1)$ stalling on $G_2=0$), so the question is whether any problem exhibits this stall for all choices.
Extended reading notes
Core claim
The central claim is that a generalized successive continuation paradigm solves constrained optimization problems with simultaneous equality and inequality constraints by seeking roots of an augmented system that includes the original equations, adjoint conditions linear and homogeneous in the Lagrange multipliers, and relaxed complementarity conditions of the form $\chi(\sigma_i,-G_i(u))=\kappa_i$, with $\chi(a,b)=\sqrt{a^2+b^2}-a-b$. Starting from a solution of the equality constraints with all multipliers set to zero and with $\kappa_i=0$ for inactive inequalities and $\kappa_i>0$ for violated ones, the algorithm follows one-dimensional solution manifolds, detects fold points or singular points, switches branches to manifolds on which multipliers become nonzero, and finally drives the relaxation parameters $\kappa_i$ (and normalization parameters $\nu$) to their KKT limits. The paper's lemmas establish, under rank conditions, that the necessary branch points exist and that the terminal point of the sequence satisfies the KKT conditions. Consequently, the search can begin from a potentially infeasible initial guess and requires no nonzero multiplier initialization.
Load-bearing premise
Each continuation stage must actually be able to drive its target parameter ($\nu$ components to 1, $\kappa$ components to 0) within the chosen computational domain, which may require successfully bypassing singular points; the paper does not prove this reachability for all problems.
Editorial extensions
If this is right
- For a problem with $d$ unknown design or control variables, the staged construction reaches a KKT point in $d+1$ continuation runs (or $d$ runs when the first stage uses branch switching), independent of the number of constraints.
- Zero Lagrange multipliers and an infeasible initial point are enough to start the search; the method does not require a feasible starting guess.
- Inequality constraints add new opportunities for branch switching, so they can help rather than merely obstruct the continuation search.
- The formulation is compatible with staged-construction continuation software that generates adjoints automatically, so it applies to two-point boundary-value problems and optimal control problems with integral inequality constraints.
- If a secondary branch is negatively aligned at a singular point, the standard predictor-corrector continuation algorithm may fail there; the paper notes that manual switching or an alternative initial choice and ordering of stages is then needed.
Reading between the lines
- A natural test is to replace the nonsmooth complementarity function by a smooth approximant and add a final continuation stage that drives the smoothing parameter to zero; this could eliminate many of the singular points that force manual branch switching, an extension the paper itself raises.
- Because the order in which components of $\nu$ and $\kappa$ are driven to their limits is chosen by the user, different orders may converge to different local optima; a systematic ordering rule could be benchmarked on multi-extremum problems.
- The framework is stated for finite-dimensional inequality constraints, so a meaningful extension is to discretize infinite-dimensional inequality constraints in a way consistent with the adjoint discretization and test whether the staged convergence still reaches KKT points.
- The same complementarity relaxation could be applied to regularize bang-bang optimal control problems, treating the singular limit as the terminal continuation stage; the paper mentions such regularization as a possible combination.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper generalizes the successive continuation paradigm of Kernévez and Doedel to constrained optimization problems with simultaneous equality and inequality constraints. The authors augment the KKT conditions with the Fischer-Burmeister complementarity function, define a family of restricted continuation problems, and use staged continuation to drive relaxation parameters to their limiting values. They prove several local lemmas (Appendix A), implement the method by extending the COCO software package, and demonstrate it on a finite-dimensional example, a two-point boundary-value problem with an integral inequality constraint, and an optimal control problem. The central claim is that optima can be found from easily initialized stages, without seeding nonzero Lagrange multipliers in the first stage, and starting from points that may violate the inequality constraints.
Significance. If the central claim is fully established, the paper offers a genuinely useful extension of a known continuation-based optimization technique, with potential applications to boundary-value and optimal control problems. The authors provide analytic and numerical evidence for the finite-dimensional motivating example, a working software implementation, and explicit discussion of the method's limitations, which is commendable. The local lemmas in Appendix A are plausible and provide a useful framework for understanding branch points and secondary branches. However, the paper's headline claim about a successful search from infeasible initial guesses is not fully supported by the theoretical analysis, because the global reachability of each continuation stage is assumed rather than proved, and the numerical examples themselves reveal multiple failures that require manual intervention or fortuitous branch alignment.
major comments (3)
- [Section 7 and Sections 3, 6.1] The paper's own concluding section admits that the success of the method depends on an unproved assumption: 'It has been tacitly assumed that each successive stage of continuation is able to drive the appropriate continuation parameters to their desired values, preferably monotonically. But the examples showed that this may not be possible within a given computational domain, or may only be possible by occasionally bearing in a direction away from the desired values.' This is a load-bearing assumption for the abstract's claim that 'a successful search for optima is found to be possible also from an infeasible initial solution guess,' because every stage of the algorithm requires reaching a target value of a component of ν or κ. The finite-dimensional example in Section 3 explicitly fails for the initial point u0 in U+/-: continuation terminates at the singular point (7/5, 7/5) on G2=0, the secondary branch is negatively aligned, and the authors restart from a different infeasible point. Similarly, in Section 6.1 the run starting from FP2 fails to reach νp2=0, terminating at a singular point on G_int=0, and the authors note that success would require a larger computational domain and continuation in the other direction. These failures are not isolated artifacts: they concern the central mechanism by which the method is supposed to reach a KKT point.
- [Appendix A, Lemma A.6 and Section 4, paragraph 5] Lemma A.6 establishes only a local one-dimensional manifold on the zero-level surface of G_k, with σ_k and σ_P nonzero for η1 close to 0. It does not provide any criterion ensuring that the secondary branch can be followed to η1=1, nor that the tangent directions at the singular point are positively aligned so that the pseudo-arclength algorithm can automatically bypass the singularity. The text in Section 4 states 'Suppose that no element of G_Z equals 0 along this manifold for η1∈[0,1]' and 'Lemma A.6 allows for the possibility of branch switching,' but these suppositions are not derived from the lemmas. The example in Section 3 shows exactly the opposite: at the singular point on G2=0 the secondary branch is negatively aligned and the pseudo-arclength algorithm cannot bypass it. Without a global reachability or alignment condition, the proposed procedure remains a heuristic with local justification, not a rigorous guarantee of successful search.
- [Section 4, 'Initialization' bullet and Section 7, first paragraph] The paper describes a 'rigorous framework' in the conclusions, but the lemmas in Appendix A are proofs by sketch relying on 'generically' and continuity arguments (e.g., Lemma A.3), and they do not quantify how far the secondary branches can be continued. Moreover, the algorithm requires the user to choose the initial point u0, the index sets I and P, the continuation order, and the computational domain; the paper itself states 'We are not able to propose a systematic selection algorithm beyond the principles outlined above.' While user choices are common in continuation methods, the combination of an unproved reachability assumption and the acknowledged need for manual switching or restarts in several examples means the manuscript overstates the degree to which it delivers a reliable search method. The claims should either be weakened to describe the method as a set of locally justified heuristics, or supplemented with additional conditions under which the staged continuation is guaranteed to reach a KKT point.
minor comments (4)
- [Section 2, Eq. (2.7)-(2.10)] The singularity of the Fischer-Burmeister function at (0,0) is identified, and the paper correctly assumes that no active inequality constraints are present at the initial point in the lemmas. The notation κ0 and the role of the set P would benefit from being summarized in a table for readability, but this is not essential.
- [Section 3, numerical results paragraphs] The numerical results for the finite-dimensional example are presented through multiple figures with color-coded markers. In the printed text, the colors are referenced (e.g., 'red dots,' 'blue dots'), but a reader without access to the color figures may find it difficult to follow the description. Consider referencing markers by shape or adding a short legend in each figure.
- [Section 6.1, first case] The text says 'two fold points in the value of κint are encountered on the way to 0 in the final continuation run' but does not explain how the continuation handles these folds. Since the paper's main algorithm relies on the ability to drive κ to zero, a brief remark about the handling of folds would be useful.
- [General] The paper frequently refers to reference [18] for implementation details of adjoint construction and for previous observations about non-monotone continuation. Since [18] is the authors' own work, the dependence is understandable, but the manuscript should make clear which specific claims rely on [18] and which are new in this paper.
Circularity Check
No significant circularity: the continuation construction is self-contained, the finite-dimensional results are checked against independent analytical solutions, and the cited self-work is only an implementation dependency.
full rationale
The paper's derivation chain is self-contained. The augmented continuation system (4.1) is constructed directly from the KKT conditions (2.2)-(2.4), with the Fischer-Burmeister function used only to rewrite complementarity conditions as equations via the equivalence (2.5)-(2.7); no fitted parameter is later renamed as a prediction. The Appendix A lemmas are local implicit-function and branch-point arguments whose hypotheses (full-rank Fredholm maps, generic transversality) are stated independently of the target stationary point, and they are proved rather than imported from prior work. In Section 3, the optimum u=(5/3,1/3) is first obtained analytically by solving the KKT conditions (3.2), and the continuation results are explicitly reported as 'consistent with the analytical solution,' so the numerical computation is not used to define the target of the derivation. The only author-relevant citation, [18], concerns the COCO staged-construction implementation and adjoint library; it is invoked for software compatibility and discretization details, not to justify the mathematical existence of the continuation stages. Section 7's concession that it was 'tacitly assumed that each successive stage of continuation is able to drive the appropriate continuation parameters to their desired values' identifies a real reachability limitation and correctness risk, but it is not a circular step: the claimed output is not equivalent by construction to an input of the method. No step satisfying the quote-and-reduction standard for circularity was found.
Assumptions & free parameters
free parameters (4)
- Initial guess u0 =
e.g., (3,1), (−4,0), (1,2), (1,−2), (0,0,0.1,0), p=0
- Index sets I and P =
e.g., I={4}, P={1}; I={3,4}, P=∅; various in Section 6
- Computational domain bounds =
e.g., −0.2≤p1≤3.5, −0.2≤p2≤1.5, −0.2≤p3≤1.0, 0≤J≤1.5 in Section 6.1
- Continuation order =
e.g., drive νJ to 1, then νp3, then κint; or νJ, νp2, νp3
assumptions (5)
- domain assumption Constraint qualification (range condition) at the optimum
- standard math Fischer-Burmeister NCP function properties
- ad hoc to paper Transversality and genericity conditions in Lemmas A.1 through A.6
- domain assumption Reachability of continuation targets
- domain assumption Discretization and adjoint consistency
Cite this review
Pith. "Pith review of Optimization with Equality and Inequality Constraints Using Parameter Continuation." pith.science (2026). https://pith.science/paper/OB2HQVXR
@misc{pith2026190901422,
author = {Pith},
title = {Pith review of: Optimization with Equality and Inequality Constraints Using Parameter Continuation},
year = {2026},
howpublished = {\url{https://pith.science/paper/OB2HQVXR}},
note = {Machine review of arXiv:1909.01422}
}
read the original abstract
We generalize the successive continuation paradigm introduced by Kern\'evez and Doedel [16] for locating locally optimal solutions of constrained optimization problems to the case of simultaneous equality and inequality constraints. The analysis shows that potential optima may be found at the end of a sequence of easily-initialized separate stages of continuation, without the need to seed the first stage of continuation with nonzero values for the corresponding Lagrange multipliers. A key enabler of the proposed generalization is the use of complementarity functions to define relaxed complementary conditions, followed by the use of continuation to arrive at the limit required by the Karush-Kuhn-Tucker theory. As a result, a successful search for optima is found to be possible also from an infeasible initial solution guess. The discussion shows that the proposed paradigm is compatible with the staged construction approach of the COCO software package. This is evidenced by a modified form of the COCO core used to produce the numerical results reported here. These illustrate the efficacy of the continuation approach in locating stationary solutions of an objective function along families of two-point boundary value problems and in optimal control problems.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
A. Ben-Tal and J. Zowe, A unified theory of first and second order conditions for extremum problems in topological vector spaces, in Optimality and Stability in Mathematical Programming, Springer, 1982, pp. 39–76, https://doi.org/10.1007/BFb0120982
-
[2]
R. Bertrand and R. Epenoy , New smoothing techniques for solving bang–bang optimal control prob- lemsnumerical results and statistical interpretation , Optimal Control Applications and Methods, 23 (2002), pp. 171–197, https://doi.org/10.1002/oca.709
doi:10.1002/oca.709 2002
-
[3]
S. C. Billups and K. G. Murty , Complementarity problems , J. Comput. Appl. Math., 124 (2000), pp. 303–318, https://doi.org/10.1016/S0377-0427(00)00432-5
-
[4]
A. E. Bryson and Y.-C. Ho , Applied Optimal Control, Hemisphere, Washington, DC, 1975
work page 1975
-
[5]
H. Dankowicz and F. Schilder , Recipes for Continuation, SIAM, 2013. CONSTRAINED OPTIMIZATION USING PARAMETER CONTINUATION 27
work page 2013
-
[6]
G. D’Avino, S. Crescitelli, P. Maffettone, and M. Grosso, A critical appraisal of the π-criterion through continuation/optimization, Chem. Eng. Sci., 61 (2006), pp. 4689–4696, https://doi.org/10. 1016/j.ces.2006.02.024
work page 2006
-
[7]
G. D’Avino, S. Crescitelli, P. Maffettone, and M. Grosso , On the choice of the optimal periodic operation for a continuous fermentation process , Biotechnol. Prog., 26 (2010), pp. 1580–1589, https: //doi.org/10.1002/btpr.461
- [8]
Show all 24 references
-
[9]
Doedel, H
E. Doedel, H. B. Keller, and J. P. Kern´evez, Numerical analysis and control of bifurcation problems (ii): Bifurcation in infinite dimensions , Internat. J. Bifur. Chaos Appl. Sci. Engrg., 1 (1991), pp. 745– 772, https://doi.org/10.1142/S0218127491000555
1991 doi
-
[10]
B. C. Fabien , Indirect solution of inequality constrained and singular optimal control problems via a simple continuation method, J. Dyn. Syst. Meas. Control, 136 (2014), p. 021003, https://doi.org/10. 1115/1.4025596
2014
-
[11]
P. E. Farrell, D. A. Ham, S. W. Funke, and M. E. Rognes , Automated derivation of the adjoint of high-level transient finite element programs , SIAM Journal on Scientific Computing, 35 (2013), pp. C369–C393
2013
-
[12]
M. C. Ferris and J.-S. Pang , Engineering and economic applications of complementarity problems , SIAM Review, 39 (1997), pp. 669–713, https://doi.org/10.1137/S0036144595285963
1997 doi
-
[13]
Gerdts , Global convergence of a nonsmooth newton method for control-state constrained optimal control problems, SIAM J
M. Gerdts , Global convergence of a nonsmooth newton method for control-state constrained optimal control problems, SIAM J. Optim., 19 (2008), pp. 326–350, https://doi.org/10.1137/060657546
2008 doi
-
[14]
A. C. Hindmarsh, P. N. Brown, K. E. Grant, S. L. Lee, R. Serban, D. E. Shumaker, and C. S. Woodward, SUNDIALS: Suite of nonlinear and differential/algebraic equation solvers , ACM Transactions on Mathematical Software (TOMS), 31 (2005), pp. 363–396
2005
-
[15]
Jiang, H
F. Jiang, H. Baoyin, and J. Li , Practical techniques for low-thrust trajectory optimization with ho- motopic approach , Journal of Guidance, Control, and Dynamics, 35 (2012), pp. 245–258, https: //doi.org/10.2514/1.52476
2012 doi
-
[16]
Kern ´evez and E
J. Kern ´evez and E. Doedel , Optimization in bifurcation problems using a continuation method , in Bifurcation: Analysis, Algorithms, Applications, Springer, 1987, pp. 153–160, https://doi.org/10. 1007/978-3-0348-7241-6 16
1987
-
[17]
Lau ß, S
T. Lau ß, S. Oberpeilsteiner, W. Steiner, and K. Nachbagauer , The discrete adjoint gradient computation for optimization problems in multibody dynamics , J. Comput. Nonlinear Dynam., 12 (2017), 031016, https://doi.org/10.1115/1.4035197
2017 doi
-
[18]
Li and H
M. Li and H. Dankowicz , Staged construction of adjoints for constrained optimization of integro- differential boundary-value problems , SIAM J. Appl. Dyn. Syst., 17 (2018), pp. 1117–1151, https: //doi.org/10.1137/17M1143563
2018 doi
-
[19]
L. B. Prasad, B. Tyagi, and H. O. Gupta, Optimal control of nonlinear inverted pendulum dynamical system with disturbance input using PID controller & LQR , in Control System, Computing and Engineering (ICCSCE), 2011 IEEE International Conference on, IEEE, 2011, pp. 540–545, h...
2011
-
[20]
Schilder, H
F. Schilder, H. Dankowicz, and M. Li , coco, http://sourceforge.net/projects/cocotools (accessed 2018/11/30)
2018
-
[21]
Sun and L
D. Sun and L. Qi , On NCP-functions, Comput. Optim. Appl., 13 (1999), pp. 201–220, https://doi.org/ 10.1023/A:1008669226453
1999 doi
-
[22]
J. O. Toilliez and A. J. Szeri , Optimized translation of microbubbles driven by acoustic fields , J. Acoust. Soc. Am., 123 (2008), pp. 1916–1930, https://doi.org/10.1121/1.2887413
2008 doi
-
[23]
Wyczalkowski and A
M. Wyczalkowski and A. J. Szeri , Optimization of acoustic scattering from dual-frequency driven microbubbles at the difference frequency , J. Acoust. Soc. Am., 113 (2003), pp. 3073–3079, https: //doi.org/10.1121/1.1570442
2003 doi
-
[24]
M. J. Zahr, P.-O. Persson, and J. Wilkening , A fully discrete adjoint method for optimization of flow problems on deforming domains with time-periodicity constraints, Comput. & Fluids, 139 (2016), pp. 130–147, https://doi.org/10.1016/j.compfluid.2016.05.021
2016 doi
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.