Pith. sign in

REVIEW 3 major objections 5 minor 10 references

Revisiting Simultaneous Methods for Dynamic Optimization in the GPU Era

T0 review · 3 major / 5 minor · reviewed 2026-07-15 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read On large systems-biology parameter estimation problems, a GPU simultaneous collocation method can be up to 5.4 imes faster than sequential integration when both succeed, and the size advantage appears only on GPU.

desk verdict Solid empirical GPU head-to-head on simultaneous vs sequential for PEtab estimation; restricted 5.4× claim holds on the successful large cases, reliability gap is already stated. read the letter →

arxiv 2607.12201 v1 pith:WGYNETZU submitted 2026-07-13 math.OC

Joseph W. Choi , Sungho Shin This is my paper · ORCID
classification math.OC MSC 90C3065K0549M37
keywords dynamicoptimizationsimultaneousmethodssequentialGPUcomputingparameterestimationorthogonalcollocationsystemsbiologyDAE-constrainedNLP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reopens the old sequential-versus-simultaneous choice for DAE-constrained optimization now that GPUs can accelerate sparse linear algebra and automatic differentiation. Sequential methods keep adaptive time-stepping inside an ODE/DAE solver, which is good for stiffness but forces the optimizer to wait for a serial forward integration at every step. Simultaneous methods turn the whole trajectory into algebraic constraints on a collocation mesh, producing a large sparse NLP whose repetitive structure maps well onto SIMD-style AD and elimination-tree parallelism. The authors implement orthogonal collocation for standardized systems-biology parameter-estimation models, solve the resulting NLPs with both GPU and CPU interior-point solvers, and compare wall-clock times and solution quality against a sequential baseline that uses adaptive integrators. On the largest instances where both approaches converge, the GPU simultaneous method reaches up to 5.4 imes speedup; the speedup grows with problem size only when the linear algebra runs on the GPU. Reliability remains lower: fixed meshes, poor initialization, and stiffness still produce timeouts, restoration failures, and local minima that the sequential method avoids.

What carries the argument

Orthogonal collocation on a fixed mesh, with condition-dependent variables lifted so that the residual expressions become a small number of SIMD-parallel patterns repeated across every interval, collocation point, and experiment; the resulting sparse KKT systems are then factored by GPU sparse linear algebra.

What would settle it

On the same largest successful models, re-solve with an adaptive outer mesh-refinement loop or with a substantially finer fixed mesh; if the GPU simultaneous speedup disappears or the relative-objective-gap cases reverse, the claimed size advantage is mesh-dependent rather than inherent.

Watch

Extended reading notes

Core claim

Among the largest PEtab parameter-estimation instances on which both methods succeed, a simultaneous orthogonal-collocation formulation solved by a GPU interior-point method is up to 5.4 imes faster than a sequential adaptive-integration baseline, and the performance advantage with respect to problem size is markedly stronger on GPUs than on CPUs.

Load-bearing premise

A single fixed collocation mesh built from one forward solve at the nominal parameter guess stays accurate enough for the whole optimization path and for the true optimum.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper revisits sequential versus simultaneous methods for DAE-constrained optimization under GPU acceleration. Sequential methods retain adaptive time-stepping advantages for stiff systems but limit parallelization; simultaneous methods discretize the DAE into a large sparse NLP that can exploit SIMD automatic differentiation and GPU sparse linear algebra. The authors introduce ExaModelsPEtab.jl, which formulates orthogonal collocation NLPs from PEtab models in ExaModels.jl, and benchmark it (MadNLP GPU/CPU) against a sequential PEtab.jl baseline on 20 systems-biology parameter-estimation problems. On the subset of largest instances where both methods succeed, the simultaneous GPU approach achieves up to 5.4 imes wall-time speedup, and the size-dependent advantage is more pronounced on GPU than on CPU; reliability remains lower due to fixed-mesh and initialization issues.

Significance. The work supplies a timely empirical comparison of a classical algorithmic dichotomy in the GPU era, together with a reusable open-source package (ExaModelsPEtab.jl) and a fully version-pinned, publicly reproducible benchmark stack. The central claim is carefully restricted to successful large instances and is supported by concrete wall times, statuses, and relative objective gaps (Table 1, Figure 1). The explicit documentation of failure modes (fixed-mesh loss of accuracy, restoration failures, kernel-size limits) and the open code repository strengthen the contribution for the process-systems and computational-biology communities.

major comments (3)
  1. Numerical Results / Static collocation mesh limitations: the fixed mesh and state guesses are generated by a single forward solve at the nominal θ₀. The paper itself states that this accuracy guarantee is lost as θ moves away from θ₀ and that stiffness can produce false optima (large ROG). Because the abstract and conclusion still present the simultaneous method as competitive once a “sufficiently fine initial mesh” is chosen, the manuscript needs either (i) a quantitative mesh-sensitivity study (e.g., successive refinement of N or K on the failed/suboptimal models) or (ii) a clearer statement that the reported speedups apply only under the fixed-mesh heuristic and that adaptive refinement remains an open GPU-compatible research question.
  2. Benchmark setup / Selected benchmark models: only 20 of 35 PEtab models are retained (15 excluded for discontinuities or PEtab.jl compilation failures). The abstract’s “up to 5.4 imes” claim and the size-trend statement rest on the successful subset of these 20. A short appendix or supplementary table listing the excluded models and the precise exclusion criteria would allow readers to assess selection bias; without it the generalizability of the reliability gap remains hard to judge.
  3. Optimization framework configurations: the sequential baseline reports the best of three optimizers (IPNewton / Fides-GN / Fides-BFGS) chosen by a model-size heuristic, while the simultaneous side uses a single interior-point solver with fixed KKT tolerances. This “best-of-three” versus “single-solver” design is acknowledged but still weakens direct head-to-head claims. Either fix one sequential optimizer for all models or report all three sequential times so that the speedup distribution can be inspected under a common algorithmic family.
minor comments (5)
  1. Abstract and Table 1: the 5.4 imes figure is not immediately recoverable from the tabulated t_SGM values; either annotate the corresponding model(s) in the text or add a column of speedup ratios.
  2. Figure 1 caption: clarify that only status 0/0A points are plotted and that the x-axis is the simultaneous NLP size (nvar), not the original ODE dimension.
  3. Implementation of ExaModelsPEtab: the code listing for collocation kernels is helpful; a one-sentence note on how condition-dependent parameters p^c are lifted would improve reproducibility for readers unfamiliar with ExaModels.
  4. Trivial solves (Boehm, Fiedler): the discussion is useful; consider moving the quantitative impact of these artifacts into a short footnote so that the geometric-mean statements remain transparent.
  5. References: InfiniteExaModels.jl and KIPET are correctly cited; a brief remark on how the present specialization of GPU kernels differs from InfiniteExaModels would sharpen the contribution statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical methods comparison with external sequential baseline and independent objective evaluation.

full rationale

This paper is an empirical performance comparison of sequential (PEtab.jl) versus simultaneous (ExaModelsPEtab.jl + MadNLP) formulations for DAE-constrained parameter estimation, with and without GPU acceleration. The central claims—up to 5.4× wall-time speedup on the largest instances where both methods succeed, and a more favorable size-scaling trend on GPU than CPU—are measured quantities (shifted geometric mean solve times, ROG against the sequential objective evaluator Φ_seq,PEtab). They are not derived from first principles, uniqueness theorems, or fitted free parameters that are then re-labeled as predictions. Self-citations to ExaModels.jl / MadNLP.jl / LiftedKKT are tooling references that enable the simultaneous GPU path; they do not force the reported speedups by construction. The fixed-mesh heuristic (mesh and state guesses from one forward solve at nominal θ₀) is a modeling premise whose limitations the authors themselves document under “Static collocation mesh limitations,” and the abstract already conditions the speedup claim on successful solves. No step reduces Eq. X to Eq. Y by definition, renames a known empirical pattern, or imports a load-bearing uniqueness result from overlapping authors. Score 0 is therefore the correct, proportionate finding.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central claim is empirical (observed wall-clock ratios on a fixed benchmark suite under stated solver settings). It rests on standard collocation theory, the correctness of the ExaModels/MadNLP GPU stack, and the modeling choice that a single forward-solved mesh at nominal θ₀ plus fixed K=4 is adequate. No free parameters are fitted to produce the speedup numbers; K and tolerances are hand-chosen configuration constants.

free parameters (2)
  • collocation polynomial degree K = 4
    Hand-chosen as K=4 after internal experiments for 'most consistency between performance and collocation accuracy'; not derived from theory for the benchmark set.
  • MadNLP KKT and acceptable tolerances = 1e-6 / 1e-4
    Relative KKT 1e-6 and acceptable 1e-4 within 15 iterations, chosen to roughly match PEtab.jl gradient tolerance; configuration knobs that affect reported solve times and statuses.
assumptions (4)
  • standard math Orthogonal collocation with Gauss-Legendre roots and Lagrange basis of degree K yields a consistent algebraic transcription of the ODE-constrained estimation problem.
    Standard simultaneous-method theory (Biegler 2010) used to write eqs. (2)–(3).
  • domain assumption A fixed mesh obtained by integrating at the nominal parameter guess θ₀ remains sufficiently accurate for optimization trajectories near the reported optima.
    Stated as the mesh-generation heuristic; the paper later notes the guarantee is lost away from θ₀ (Static collocation mesh limitations).
  • domain assumption Lifting condition-dependent parameters into the RHS input space reduces unique expression patterns to Nz, enabling SIMD AD without changing the mathematical problem.
    Core design choice in Implementation of ExaModelsPEtab; correctness is standard but the performance claim depends on it.
  • domain assumption cuDSS / LiftedKKT on GPU and ma57 on CPU correctly solve the MadNLP KKT systems to the stated tolerances.
    Solver correctness is assumed from prior MadNLP/ExaModels work; not re-proved here.
invented entities (1)
  • ExaModelsPEtab.jl independent evidence
    purpose: GPU-compatible automatic simultaneous transcription of PEtab models into ExaModels kernels with condition lifting.
    Software artifact introduced by the paper; not a physical or mathematical entity, but the vehicle for the empirical claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Simultaneous Methods for Dynamic Optimization in the GPU Era." pith.science (2026). https://pith.science/paper/WGYNETZU

@misc{pith2026260712201,
  author       = {Pith},
  title        = {Pith review of: Revisiting Simultaneous Methods for Dynamic Optimization in the GPU Era},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WGYNETZU}},
  note         = {Machine review of arXiv:2607.12201}
}
read the original abstract

We revisit the classical topic in dynamic optimization: sequential vs simultaneous methods for solving DAE-constrained optimization problems, with a particular focus on how graphics processing unit (GPU) computing changes their effectiveness. Sequential methods offer key advantages through adaptive time stepping at the differential-algebraic equation (DAE) solver level, which is especially effective for handling stiff systems. However, long-time-horizon simulations remain a computational bottleneck, as time integration is inherently sequential and limits parallelization within the optimization algorithm. In contrast, simultaneous approaches are well-suited for parallel computing. They address these limitations by exploiting the highly repetitive structure of discretized DAE systems at the function evaluation level and leveraging sparse linear algebra routines that enable elimination tree-level parallelism. Although simultaneous methods typically lack adaptive time stepping, this limitation can often be mitigated by choosing a sufficiently fine initial mesh or iteratively adjusting mesh coarseness in an outer loop. In this work, we revisit the capabilities of the simultaneous approach in a GPU computing environment and assess its performance against a sequential method baseline. We employ a simultaneous approach based on orthogonal collocation within an open-source modeling framework and apply it to parameter estimation benchmarks from systems biology. We evaluate both GPU and CPU solvers on the simultaneous formulation. Our results show that, although less reliable, the simultaneous approach achieves up to 5.4x speedup compared to the sequential baseline among the largest instances where both methods solve successfully. The advantage of the simultaneous method with respect to problem size is more pronounced on GPUs than on CPUs.

Figures

Figures reproduced from arXiv: 2607.12201 by the authors.

Figure 1
Figure 1. Speedup of ExaModelsPEtab.jl (GPU/CPU) vs PEtab.jl based on simultaneous method NLP size for status 0/0A solves. PEtab.jl model x-axis position corresponds to its ExaModels counterpart of that size. Static collocation mesh limitations Although our fixed-mesh heuristic guarantees an accurate state profile at the nominal guess 𝜃0 , this guar￾antee is lost as 𝜃 moves away from 𝜃0 . The initialized col￾location mesh may… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 1 linked inside Pith

  1. [1]

    Nonlinear Programming: Concepts, Algorithms, and Applications to Chemical Processes

    Biegler LT. Nonlinear Programming: Concepts, Algorithms, and Applications to Chemical Processes. SIAM (2010)

    2010

  2. [2]

    GPU accelerated dynamic nonlinear optimization with ExaModels and MadNLP

    Pacaud F, Shin S. GPU accelerated dynamic nonlinear optimization with ExaModels and MadNLP. IEEE 63rd CDC (2024)

    2024

  3. [3]

    The PEtab Benchmark Collection of parameter estimation problems

    The PEtab Benchmark Collection Contributors. The PEtab Benchmark Collection of parameter estimation problems. Version v2026.01.24 (2026)

    2026

  4. [4]

    PEtab—Interoperable specification of parameter estimation problems in systems biology

    Schmiester L, Schälte Y, Bergmann FT, Camba T, Dudkin E, Egert J, Fröhlich F, Fuhrmann L, Hauber AL, Kemmer S, et al. PEtab—Interoperable specification of parameter estimation problems in systems biology. PLoS Comput Biol 17.1:e1008646 (2021)

    2021

  5. [5]

    PEtab.jl: advancing the efficiency and utility of dynamic modelling

    Persson S, Fröhlich F, Grein S, Loman T, Ognissanti D, Hasselgren V, Hasenauer J, Cvijovic M. PEtab.jl: advancing the efficiency and utility of dynamic modelling. Bioinformatics 41.9:btaf497 (2025)

    2025

  6. [6]

    Advances to modeling and solving infinite-dimensional optimization problems in InfiniteOpt.jl

    Gondosiswanto E, Pulsipher JL. Advances to modeling and solving infinite-dimensional optimization problems in InfiniteOpt.jl. Digit Chem Eng 15:100236 (2025)

    2025

  7. [7]

    A unifying modeling abstraction for infinite- dimensional optimization

    Pulsipher JL, Zhang W, Hongisto TJ, Zavala VM. A unifying modeling abstraction for infinite- dimensional optimization. Comput Chem Eng 156 (2022)

    2022

  8. [8]

    KIPET–an open-source kinetic parameter estimation toolkit

    Short M, Schenk C, Thierry D, Rodriguez JS, Biegler LT, Garcia-Muñoz S. KIPET–an open-source kinetic parameter estimation toolkit. Comput Aided Chem Eng 47:299-304 (2019)

    2019

Show all 10 references
  1. [9]

    pyomo.dae: a modeling and automatic discretization framework for optimization with differential and algebraic equations

    Nicholson B, Siirola JD, Watson JP, Zavala VM, Biegler LT. pyomo.dae: a modeling and automatic discretization framework for optimization with differential and algebraic equations. Math Program Comput 10.2:187-223 (2018)

    2018

  2. [10]

    Condensed interior-point methods: porting reduced-space approaches on GPU hardware

    Pacaud F, Shin S, Schanen M, Maldonado DA, Anitescu M. Condensed interior-point methods: porting reduced-space approaches on GPU hardware. arXiv preprint arXiv:2203.11875 (2022) © 2027 by the authors. Licensed to PSEcommunity.org and PSE Press. This is an open access article u...

    2022 arXiv

Pith tools

Reviewed July 15, 2026 · model on record in the stance chip above.