Exact Decomposition of Adversarial Dual-Objective Value Functions, with Applications to Optimal Drug Dosing

Dylan Hirsch; Sylvia Herbert; William Sharpless

Exact Decomposition of Adversarial Dual-Objective Value Functions, with Applications to Optimal Drug Dosing

T0 review · 0 major / 3 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that two composite Hamilton-Jacobi reachability value functions — reach-always-avoid and reach-reach — decompose exactly into basic reach/avoid value functions even when an adversarial disturbance is present, and demonstra

desk verdict Genuinely new adversarial decomposition theorems for RAA and RR value functions in continuous-time HJR; the proofs hold up, and the counterexample motivates the results. read the letter →

arxiv 2607.14023 v1 pith:OZZGZDQE submitted 2026-07-15 eess.SY cs.SYq-bio.QM

Dylan Hirsch , William Sharpless , Sylvia Herbert This is my paper · ORCID

classification eess.SYcs.SYq-bio.QM MSC 49N7035D40

keywords Hamilton-Jacobireachabilityvaluefunctiondecompositionnon-anticipativestrategiesreach-always-avoidreach-reachadversarialgamesoptimaldrugdosingviscositysolutions

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Classic Hamilton-Jacobi reachability gives value functions for reach, avoid, and reach-avoid tasks, but more complex dual-objective tasks such as 'reach the target and then always avoid the obstacle' (RAA) or 'reach both targets in either order' (RR) have had no guarantee that their values factor through the basic solvers once an adversarial disturbance is present. This paper proves that two particular decompositions do hold exactly for continuous-time, finite-horizon systems with non-anticipative adversary strategies: RAA reduces to an avoid value followed by a standard reach-avoid value, and RR reduces to two reach values combined into a single reach problem. The proofs construct non-anticipative strategies that switch at the first time the primal trajectory crosses a value-derived threshold. A companion counterexample shows that not every natural decomposition survives an adversary, highlighting why this certification matters. The authors then demonstrate that the RAA split removes post-therapy toxicity in a pharmacokinetic dosing example and that the RR split enables coordinated target achievement in a mutual-inhibition protein model.

What carries the argument

The key machinery is the concatenation of non-anticipative strategies at a value-function crossing time. The paper defines, for each control signal u, a switch set S_u on which the running reward r exceeds the appropriate inner value function (V_A for RAA, V_R of the other target for RR), picks s_u = min S_u (with s_u = T if empty), and then builds a composite adversary strategy that plays the primary strategy before s_u and a pre-selected family of secondary strategies after s_u. Continuity of the value functions ensures the switch sets are closed, hence have a minimum; the epsilon-chain argument then shows both directions of the equality. For the RR problem, a second switch set handles the

What would settle it

Run a dense-grid HJI comparison on the two-compartment PK model (Eq. 4-6) with zero disturbance: if V_RAA[r,q] and V_RA[min{r,V_A[q]},q] differ by more than solver tolerance at any grid point, the identity fails; the same check on a system where V_A[q] is artificially modified to be discontinuous would test the continuity requirement.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a pair of identities: V_RAA[r,q] = V_RA[min{r, V_A[q]}, q] and V_RR[r1,r2] = V_R[max{min{r1,V_R[r2]}, min{r2,V_R[r1]}}], both for continuous r, q, r1, r2 and t < T. The authors prove each identity in two inclusions, using epsilon-optimal strategies: for one inequality they concatenate an adversary strategy for the primal objective with a family of near-optimal strategies for the secondary objective, switching at the first time the target reward reaches the secondary value function; for the other inequality they splice a control signal after an arbitrarily chosen switch time. The paper also exhibits a counterexample (order-constrained reach)

Load-bearing premise

The entire proof rests on being able to define (and keep non-anticipative) a composite adversary strategy that switches at the minimum time the running reward crosses the inner value function; that requires the inner value functions to be continuous and the switch-time map to be measurable, which the paper justifies only by citing standard value-function continuity and sketching the non-anticipativity check.

Editorial extensions

If this is right

Computing V_RAA becomes a two-stage pipeline: solve an avoid value, form the modified target, then solve a standard reach-avoid; no dedicated RAA HJI PDE is needed.
Computing V_RR likewise reduces to two reach solves and one final reach solve, with the modified target combining the two reach values.
The switching rule from the proof yields a natural closed-loop controller: drive with the composite value until the running reward crosses the secondary value, then hand off to the secondary value's controller.
The counterexample for ordered-reach shows the decomposition is not a formal free-for-all: only the specific min/max structures of RAA and RR are certified to survive an adversary.
In the drug-dosing examples, RAA-based policies keep kidney concentration below the toxic threshold even after the therapeutic goal is met, which reach-avoid policies fail to do.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same switch-time argument suggests a general recipe: a composite minimax value decomposes whenever the inner value functions are continuous and act as absorbing 'barriers,' so the adversary cannot improve by crossing them; this could certify decompositions for richer temporal-logic specifications such as reach-avoid-reach or nested reach-avoid.
Because the proof's switch time is defined through exact value functions, approximate learning-based surrogates for V_A or V_R would shift the switch point and may break the guarantee; testing the sensitivity of RAA/RR closed-loop performance to value-function error would be a direct experimental extension.
The two identities are composable: applying Theorem 1 then Theorem 2 (or vice versa) may yield exact factors for a three-objective task such as 'reach target 1, then reach target 2, never entering obstacle,' which is not discussed in the paper.
The PK example treats continuous dosing; translating to discrete pill regimens will interact with the decomposition's exact switch time, a practical question left open.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: this paper proves exact decompositions for two composite HJR value functions—reach-always-avoid and reach-reach—that survive an adversarial disturbance player in continuous time. That is genuinely new relative to [7] (adversary-free) and [15] (finite-state, discrete-time, single-player). The counterexample in Section III-B shows the naive ordered RR decomposition breaks with a disturbance, so the positive results are not empty. The main theorems are proved carefully in the appendix; the epsilon-chains check out and the strategy concatenation arguments are sound. The Theorem 2 non-anticipativity claim is sketched "for space," but the pattern is the same as Theorem 1 and the sketch is convincing enough; the equal-switch-time argument goes through using continuity of V_R and V_A. The drug-dosing examples show the intended use but are illustrative—no code or data for the figures, which is a minor reproducibility gap, not a flaw in the theory.

Soft spots, in proportion: the applications section reads as motivation rather than a validation of the theorems; the simulations are not reproducible from the text. The proof of Theorem 2's (≥) direction relies on Lemma 1 and a WLOG ordering argument; both are fine. The main potential worry—whether the concatenated strategies are genuinely non-anticipative—does not land: the switch sets are closed under continuity, the minimum exists if nonempty, and non-anticipativity is checked pointwise, so no measurability issue arises. That was my first instinct too, but reading the appendix carefully it holds.

Who is this for? Researchers in HJR or safe control who want to compute RAA or RR values from standard reach/avoid solvers, or those building on single-player decomposition results. It deserves a serious referee: the result is nontrivial, the proofs are mostly complete, and the connection to prior work is honest. The citations are appropriate; self-citations [7] and [15] are the direct antecedents, not padding.

Recommendation: send it to peer review. If you referee it, ask for the omitted non-anticipativity details in Theorem 2 and for code or at least parameter tables for the simulations. But the central claim is solid.

Referee Report

0 major / 3 minor

Summary. The paper studies exact value-function decompositions for two dual-objective Hamilton-Jacobi reachability (HJR) tasks in the presence of an adversarial disturbance player, in a continuous-time, finite-horizon setting with non-anticipative adversary strategies. Theorem 1 claims that the reach-always-avoid (RAA) value function decomposes as V_RAA[r,q] = V_RA[min{r,V_A[q]}, q]. Theorem 2 claims that the reach-reach (RR) value function decomposes as V_RR[r1,r2] = V_R[max{min{r1,V_R[r2]}, min{r2,V_R[r1]}}]. The proofs in the appendix construct epsilon-optimal concatenated adversary strategies, switching between component-wise strategies at times determined by comparing the running rewards with the relevant basic value functions. The paper also gives two pharmacokinetic/pharmacodynamic dosing examples illustrating how the RAA and RR formulations improve over naive reach-avoid or simultaneous-reach approaches.

Significance. If the theorems are correct, they are a meaningful contribution to adversarial HJR: they certify that two composite value functions can be computed exactly by first solving standard reach, avoid, or reach-avoid problems and then composing the resulting value functions, despite the minimax structure that usually prevents such decompositions. The counterexample in Section III-B correctly shows that not every one-player decomposition survives an adversary, so the positive results are not vacuous. The appendix is careful: the epsilon-chains are detailed, the switch sets are closed due to continuity of the terminal functions and the basic value functions, and the non-anticipativity of the concatenated strategies is checked pointwise, with no measurability of the strategy map required by Definition 1. The potentially fragile point identified in the stress-test—existence of the switch-time minima and equality of switch times for inputs that agree up to a common time—does not land; continuity gives the needed minima, and the equal-switch argument is sound. The simulations are illustrative rather than exhaustive, but they do support the claimed practical benefit. Overall, this is a solid th

minor comments (3)

[Appendix, Theorem 2 proof, after Eq. (32)] The non-anticipativity verification for the composite strategy sigma is compressed to a one-paragraph outline 'for space.' Since admissibility of sigma is load-bearing for the (≤) direction, please expand this to the same level of detail as in Theorem 1, explicitly covering the two-switch-set case and the possibility that one or both switch times equal T.
[Section V, Figs. 1 and 2] No code or numerical-solver details are provided for the simulations. The theoretical claims do not depend on the simulations, but for reproducibility and to allow readers to independently check the claimed comparisons, please state the numerical scheme, grid resolution, and convergence tolerances, or provide code with the revision.
[Eqs. (12) and (30)-(31)] The convention 's_u := min S_u if S_u≠∅ and s_u := T otherwise' is clear, but the existence of the minimum when S_u is nonempty is only implicit. A one-sentence justification using continuity of r−V_A and r_i−V_R would help the reader.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: decomposition theorems are proved from the value-function definitions, not derived from their own conclusions or from load-bearing self-citations.

full rationale

The paper's central claims are Theorems 1 and 2, which assert exact decompositions of the adversarial RAA and RR value functions into basic reach/avoid value functions. The proofs in the appendix do not assume the target identities; instead they construct explicit non-anticipative adversary strategies and near-optimal controls, using only the definitions of the payoffs and the standard continuity of V_R, V_A, and V_RA (cited to the external textbook [6]). The modified target functions are built from the component value functions V_A[q] and V_R[r1], V_R[r2], not from V_RAA or V_RR themselves, so the decompositions are not true by definition. Self-citations [7] and [15] motivate the problem and supply the adversary-free formulas, but they are not used as proof ingredients; the adversarial proofs are self-contained. The continuity assumption flagged in the reader's take is a standard, externally verified hypothesis about the HJR framework, not a circularity. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The examples use the decomposition to synthesize controllers, but this is an application rather than evidence for the theorems. Overall, the derivation chain is independent of its conclusions.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

No free parameters or invented entities enter the theorems. The PK/PD parameters listed are hand-chosen simulation constants for the two illustrative examples. The axioms are standard HJR assumptions; the only nonstandard ingredient is the non-anticipative adversary framework borrowed from differential game theory.

free parameters (2)

PK model simulation constants = k1=0.2, k2=1, K1=0.1, K2=10, θther=θtox=1, ε=0.5, M(τ) schedule
Hand-chosen for illustrative Example 1 (Section V-A, Fig. 1). Not needed for Theorems 1-2.
PD model simulation constants = kb=2.5, ku=0.5, γ1=2, γ2=2.1, γ3=1.5, θther1=θther2=1, ε=0.5
Hand-chosen for illustrative Example 2 (Section V-B, Fig. 2). Not needed for Theorems 1-2.

assumptions (6)

domain assumption Continuity, linear growth, and local Lipschitz of f (Assumption 1)
Used in all proofs to get well-defined Carathéodory trajectories; stated in Section II-A.
domain assumption Compact control and disturbance sets U, D (Assumption 2)
Used for existence of near-optimal strategies and value continuity; stated in Section II-A.
domain assumption Adversary strategies restricted to non-anticipative maps (Definition 1)
Defines the continuous-time game value and is used throughout the appendix.
standard math V_R, V_A, V_RA are continuous for continuous r,q under Assumptions 1-2 (standard result, Ref [6])
Invoked in the appendix to justify switch sets S_u having minima and epsilon approximations; not proved in this paper.
standard math Existence/uniqueness of Carathéodory solutions (standard result, Ref [17])
Used to define trajectories in the proofs.
domain assumption r, q, r1, r2 are continuous (Theorem hypotheses)
Needed for continuity of the value functions and for the switch-time arguments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Exact Decomposition of Adversarial Dual-Objective Value Functions, with Applications to Optimal Drug Dosing." pith.science (2026). https://pith.science/paper/OZZGZDQE

@misc{pith2026260714023,
  author       = {Pith},
  title        = {Pith review of: Exact Decomposition of Adversarial Dual-Objective Value Functions, with Applications to Optimal Drug Dosing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZZGZDQE}},
  note         = {Machine review of arXiv:2607.14023}
}

read the original abstract

Hamilton-Jacobi Reachability (HJR) is a central framework in safe control theory. While HJR has traditionally focused on a few fundamental tasks, there is increasing interest in scaling to more complex objectives. Recent works have studied the exact decomposition of the value functions for two fundamental dual-objective tasks in the adversary-free setting. However, not all value function decompositions in HJR remain valid with an adversary. In this work, we develop theoretical approaches to certify that for these two composite value functions, the proposed decompositions still hold with an adversary. Finally, we show how these results can solve issues that arise when applying HJR to optimal drug regimen design.

Figures

Figures reproduced from arXiv: 2607.14023 by the authors.

**Figure 2.** Closed-loop simulation of (7)-(9), under controllers derived [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 2 linked inside Pith

[7]

Dual- objective reinforcement learning with novel Hamilton-Jacobi-Bellman formulations,

W. Sharpless, D. Hirsch, S. Tonkens, N. Shinde, and S. Herbert, “Dual- objective reinforcement learning with novel Hamilton-Jacobi-Bellman formulations,” 2025. (arXiv:2506.16016v2)

arXiv 2025
[15]

Bellman value decomposition for task logic in safe optimal control,

W. Sharpless, O. So, D. Hirsch, S. Herbert, and C. Fan, “Bellman value decomposition for task logic in safe optimal control,” 2026. (arXiv:2602.19532v1)

arXiv 2026
[1]

Hamilton-Jacobi reachability: A brief overview and recent advances,

S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-Jacobi reachability: A brief overview and recent advances,” in2017 IEEE Conf. Decis. Control, p. 2242–2253, IEEE, 2017

2017
[2]

A time-dependent Hamilton- Jacobi formulation of reachable sets for continuous dynamic games,

I. Mitchell, A. Bayen, and C. Tomlin, “A time-dependent Hamilton- Jacobi formulation of reachable sets for continuous dynamic games,” IEEE Trans. Autom. Control, vol. 50, no. 7, pp. 947–957, 2005

2005
[3]

Hamilton–Jacobi formulation for reach–avoid differential games,

K. Margellos and J. Lygeros, “Hamilton–Jacobi formulation for reach–avoid differential games,”IEEE Trans. Autom. Control, vol. 56, no. 8, pp. 1849–1861, 2011

2011
[4]

Reach-avoid problems with time-varying dynamics, targets and constraints,

J. F. Fisac, M. Chen, C. J. Tomlin, and S. S. Sastry, “Reach-avoid problems with time-varying dynamics, targets and constraints,” in Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control (HSCC), p. 11–20, 2015

2015
[5]

Differential games and repre- sentation formulas for solutions of Hamilton-Jacobi-Isaacs equations,

L. C. Evans and P. E. Souganidis, “Differential games and repre- sentation formulas for solutions of Hamilton-Jacobi-Isaacs equations,” Indiana University Mathematics Journal, vol. 33, no. 5, pp. 773–797, 1984

1984
[6]

Bardi and I

M. Bardi and I. Capuzzo-Dolcetta,Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhauser, 1997

1997

Show all 25 references

[8]

Reach-avoid-stabilize using admis- sible control sets,

Z. Gong, B. Li, and S. Herbert, “Reach-avoid-stabilize using admis- sible control sets,” in2025 Proc. IEEE Conf. Decis. Control (CDC), pp. 911–917, 2025

2025
[9]

Control synthesis for multiple reach-avoid tasks via Hamilton-Jacobi reachability analysis,

Y . Chen, S. Li, and X. Yin, “Control synthesis for multiple reach-avoid tasks via Hamilton-Jacobi reachability analysis,” in2025 IEEE Conf. Decis. Control (CDC), pp. 5980–5985, 2025

2025
[10]

Control synthesis for multiple reach-avoid tasks via Hamilton-Jacobi reachability analysis (arXiv version),

Y . Chen, S. Li, and X. Yin, “Control synthesis for multiple reach-avoid tasks via Hamilton-Jacobi reachability analysis (arXiv version),” 2025 (arXiv:2509.10896v1)

2025 arXiv
[11]

Signal temporal logic meets reachability: Connections and applications,

M. Chen, Q. Tam, S. C. Livingston, and M. Pavone, “Signal temporal logic meets reachability: Connections and applications,” inAlgorithmic F oundations of Robotics XIII, pp. 581–601, 2020

2020
[12]

Robust STL control synthesis under maximal disturbance sets,

J. Verhagen, L. Lindemann, and J. Tumova, “Robust STL control synthesis under maximal disturbance sets,” in2024 IEEE Conf. Decis. Control (CDC), pp. 315–321, 2024

2024
[13]

Ensuring safety for vehicle parking tasks using Hamilton-Jacobi reachability analysis,

F. J. Jiang, Y . Gao, L. Xie, and K. H. Johansson, “Ensuring safety for vehicle parking tasks using Hamilton-Jacobi reachability analysis,” in 2020 59th IEEE Conf. Decis. Control (CDC), pp. 1416–1421, 2020

2020
[14]

Guaranteed completion of complex tasks via temporal logic trees and Hamilton-Jacobi reachability,

F. J. Jiang, K. M. Arfvidsson, C. He, M. Chen, and K. H. Johansson, “Guaranteed completion of complex tasks via temporal logic trees and Hamilton-Jacobi reachability,” in2024 IEEE 63rd Proc. IEEE Conf. Decis. Control (CDC), p. 5203–5210, IEEE, Dec. 2024

2024
[16]

The existence of value in differential games of pursuit and evasion,

R. J. Elliott and N. J. Kalton, “The existence of value in differential games of pursuit and evasion,”J. Differential Equations, vol. 12, no. 3, pp. 504–523, 1972

1972
[17]

Friedman,Differential Games

A. Friedman,Differential Games. Dover Publications, 2013

2013
[18]

Robust satisfaction of temporal logic over real-valued signals,

A. Donz ´e and O. Maler, “Robust satisfaction of temporal logic over real-valued signals,” inF ormal Modeling and Analysis of Timed Systems, pp. 92–106, 2010

2010
[19]

Reactive synthesis from signal temporal logic specifications,

V . Raman, A. Donz ´e, D. Sadigh, R. M. Murray, and S. A. Seshia, “Reactive synthesis from signal temporal logic specifications,” in Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control, HSCC ’15, p. 239–248, 2015

2015
[20]

hj reachability

StanfordASL, “hj reachability.” github.com/StanfordASL/hj reachability
[21]

helperoc

HJReachability, “helperoc.” github.com/HJReachability/helperOC
[22]

Bridging Hamilton-Jacobi safety analysis and reinforcement learn- ing,

J. F. Fisac, N. F. Lugovoy, V . Rubies-Royo, S. Ghosh, and C. J. Tomlin, “Bridging Hamilton-Jacobi safety analysis and reinforcement learn- ing,” in2019 International Conference on Robotics and Automation (ICRA), pp. 8550–8556, 2019

2019
[23]

Deepreach: A deep learning approach to high-dimensional reachability,

S. Bansal and C. J. Tomlin, “Deepreach: A deep learning approach to high-dimensional reachability,” in2021 IEEE International Confer- ence on Robotics and Automation (ICRA), pp. 1817–1824, 2021

2021
[24]

How to mathematically optimize drug regimens using optimal control,

H. Moore, “How to mathematically optimize drug regimens using optimal control,”J. Pharmacokinet. Pharmacodynamics., vol. 45, no. 1, p. 127–137, 2018

2018
[25]

A differential game with two players and one target,

P. Cardaliaguet, “A differential game with two players and one target,” SIAM Journal on Control and Optimization, vol. 34, p. 1441–1460, July 1996

1996

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [7]

Dual- objective reinforcement learning with novel Hamilton-Jacobi-Bellman formulations,

W. Sharpless, D. Hirsch, S. Tonkens, N. Shinde, and S. Herbert, “Dual- objective reinforcement learning with novel Hamilton-Jacobi-Bellman formulations,” 2025. (arXiv:2506.16016v2)

arXiv 2025

[2] [15]

Bellman value decomposition for task logic in safe optimal control,

W. Sharpless, O. So, D. Hirsch, S. Herbert, and C. Fan, “Bellman value decomposition for task logic in safe optimal control,” 2026. (arXiv:2602.19532v1)

arXiv 2026

[3] [1]

Hamilton-Jacobi reachability: A brief overview and recent advances,

S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-Jacobi reachability: A brief overview and recent advances,” in2017 IEEE Conf. Decis. Control, p. 2242–2253, IEEE, 2017

2017

[4] [2]

A time-dependent Hamilton- Jacobi formulation of reachable sets for continuous dynamic games,

I. Mitchell, A. Bayen, and C. Tomlin, “A time-dependent Hamilton- Jacobi formulation of reachable sets for continuous dynamic games,” IEEE Trans. Autom. Control, vol. 50, no. 7, pp. 947–957, 2005

2005

[5] [3]

Hamilton–Jacobi formulation for reach–avoid differential games,

K. Margellos and J. Lygeros, “Hamilton–Jacobi formulation for reach–avoid differential games,”IEEE Trans. Autom. Control, vol. 56, no. 8, pp. 1849–1861, 2011

2011

[6] [4]

Reach-avoid problems with time-varying dynamics, targets and constraints,

J. F. Fisac, M. Chen, C. J. Tomlin, and S. S. Sastry, “Reach-avoid problems with time-varying dynamics, targets and constraints,” in Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control (HSCC), p. 11–20, 2015

2015

[7] [5]

Differential games and repre- sentation formulas for solutions of Hamilton-Jacobi-Isaacs equations,

L. C. Evans and P. E. Souganidis, “Differential games and repre- sentation formulas for solutions of Hamilton-Jacobi-Isaacs equations,” Indiana University Mathematics Journal, vol. 33, no. 5, pp. 773–797, 1984

1984

[8] [6]

Bardi and I

M. Bardi and I. Capuzzo-Dolcetta,Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhauser, 1997

1997