Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

On the differentiability of the value function of switched linear systems under arbitrary and controlled switching

T0 review · 4 major / 3 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper establishes that value functions of stable switched linear systems can be non-differentiable on dense subsets of the state space, even when the running cost is smooth and Lipschitz.

desk verdict Solid n=2 construction showing dense non-differentiability of value functions for switched linear systems, but the n>2 extension as written has a fixable gap in the cost function. read the letter →

arxiv 2511.20037 v2 pith:5EIJUOES submitted 2025-11-25 math.OC

Guillaume O. Berger This is my paper · ORCID
classification math.OC MSC 49L2093C3037E05
keywords switchedlinearsystemsvaluefunctionnon-differentiabilitydensesubsetBellmanequationintervalexchangemapjointspectralradiusoptimalcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks how irregular the value function of a stable switched linear system can be. It establishes that although these value functions are always Lipschitz continuous—values cannot vary faster than a fixed multiple of the distance between states—they need not be piecewise smooth: for every dimension at least two, there are examples with a smooth quadratic cost where both the optimal and worst-case value functions are non-differentiable on dense subsets (sets that pass arbitrarily close to every state). These examples are stable, with joint spectral radius below one, and can even use rational transition matrices. This matters because exact computation of such value functions—central to optimal control and reinforcement learning—cannot then be achieved by algorithms that search over piecewise-smooth or smooth templates; any exact representation must admit dense pointwise non-differentiability, even though the functions remain differentiable almost everywhere.

What carries the argument

The central mechanism is an interval exchange map: a piecewise translation of an interval, cut at a point ν, with the two pieces swapped. Here it takes the form T(θ)=θ+α for θ<ν and T(θ)=θ+β for θ≥ν on an interval of angles, and it is conjugate to an irrational rotation, so every backward orbit is dense. The scaled-rotation matrices reduce the planar dynamics to angle shifts, the homogeneity of the quadratic cost gives scaling identities, and the Bellman (dynamic programming) equation carries non-differentiability from the distinguished point ν backward along the dense orbit.

What would settle it

For the worst-case half, find two distinct continuous functions that both solve J(x)=c(x)+max_i J(A_i x) on a neighborhood of the origin and vanish at 0 for a stable switched linear system with smooth Lipschitz cost; that would break the uniqueness step used to identify the constructed function as the value function. Alternatively, for the paper's concrete two-rotation example, display a periodic backward orbit of the interval-exchange map T; the density argument rules such an orbit out.

Watch

Extended reading notes

Core claim

The central result, Theorem 3, states that for every n≥2 there is a stable switched linear system and a smooth Lipschitz cost such that the associated optimal and worst-case value functions are non-differentiable on dense subsets of R^n. The construction uses two scaled rotation matrices with angles α and α−π/2, where α/π is irrational, and the quadratic cost c(x)=x1^2+2x2^2; the joint spectral radius is 0.01, so the system is stable. The proof isolates a point on the unit circle where the optimal value function has a kink; the optimal switching dynamics act on the angle coordinate as an interval exchange map, and the backward orbit of that kink point is dense. The dynamic programming equati

Load-bearing premise

The worst-case half of the result rests on the uniqueness of continuous solutions of the dynamic-programming maximum equation with zero at the origin; if two distinct such solutions existed, the constructed function could satisfy the equation without being the true worst-case value.

Editorial extensions

If this is right

  • Exact value functions of stable switched linear systems can lie outside the class of piecewise-smooth functions, so algorithms searching that class cannot compute them exactly in finite time.
  • The non-differentiability occurs even with rational transition matrices, so it is not an artifact of specially chosen irrational data and can occur with nonzero probability for finite-precision matrices.
  • The value functions remain Lipschitz and hence differentiable almost everywhere; the dense non-differentiability set has Lebesgue measure zero, so sampling-based or generic derivative checks will miss it.
  • Locating extrema of these value functions cannot rely on gradients, because derivatives fail on a dense set.
  • The paper leaves open sufficient conditions for piecewise differentiability and identifies that as the announced next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to vary the rotation angle: when α/π is rational, the interval exchange map becomes periodic, so the same mechanism would likely yield piecewise-smooth or finitely nonsmooth value functions, isolating irrationality as the driver of dense non-smoothness.
  • Although the non-differentiability set is dense, it has measure zero; smooth function-approximation classes may still approximate the value function in norm, but pointwise exact recovery at the dense points would be impossible within finite-parameter smooth templates.
  • The proof suggests a broader correspondence between value-function irregularity and the orbit complexity of the optimal switching law: minimal or ergodic switching dynamics should produce dense non-smoothness, while periodic switching dynamics should not.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper studies the value functions of stable discrete-time switched linear systems under controlled and arbitrary switching. It first proves that if the cost is Lipschitz on a neighborhood of the origin, then both the optimal value function J* and the worst-case value function J^∘ are Lipschitz (Theorem 2). The main contribution is Theorem 3, which claims that for every n≥2 there exist rational, stable switched linear systems with C^∞, Lipschitz costs whose value functions are non-differentiable on dense subsets of R^n. The proof for n=2 uses a two-mode system of scaled rotations; the optimal switching law is related to an interval-exchange map, and the backward orbit of a special angle yields a dense non-differentiability set. The n>2 case is intended to follow by block-diagonal embedding of the 2D construction.

Significance. If the construction were fully correct, this would be a significant negative result for the exact computation of value functions of switched linear systems: it would show that no piecewise-smooth template can represent J* or J^∘ in general, despite the functions being differentiable almost everywhere. The n=2 proof is genuinely constructive, with explicit rational matrices, a transparent interval-exchange-map mechanism, and inequalities that are carefully checked. The Lipschitz theorem is clean and useful. However, the extension to n>2 contains a concrete error in the cost function, and the worst-case half of the n=2 proof has a sign error and an unverified uniqueness step. As submitted, Theorem 3 is not established beyond the 2D optimal half.

major comments (4)
  1. [§IV.A.2, Lemma 13] The cost function used for n>2 is \hat c(x)=cos^2(x_1)+2 sin^2(x_2). For x=(cosθ,sinθ,0,...,0), \hat c(\hat A_1 x)=cos^2(ρ cos(θ+α))+2 sin^2(ρ sin(θ+α)), whereas the 2D recursion with the quadratic cost c_2(u)=u_1^2+2u_2^2 requires ρ^2(cos^2(θ+α)+2 sin^2(θ+α)). These are not equal; e.g., θ=π/4, α=arctan(3/4), ρ=0.01. Hence Lemma 13 is false as stated, and the proof of Corollary 5 and Theorem 3 for n>2 fails. The intended repair—using c(x)=x_1^2+2x_2^2 in every dimension—would restore the decoupling argument, but that is not the cost written in the manuscript.
  2. [§IV.A.1, Corollary 4] The displayed derivation for \tilde J^∘ ends with 'min{...}' but the algebra, and the definition of the worst-case value, give 'max{...}'. With the printed 'min', the constructed function does not satisfy the worst-case Bellman equation (3), so the conclusion that it equals J^∘ is unsupported. The sign in the final displayed equation must be corrected.
  3. [§IV.A.1, Corollary 4] The step 'This implies that J^∘ is the worst-case value function, by classical arguments ... [19, Thm 3.1]' needs to be made precise. The paper neither states the theorem nor verifies its hypotheses, including its applicability to a supremum (rather than infimum) Bellman equation, the required growth condition on solutions, and the normalization at the origin. Without this, a continuous solution of (3) need not be the true worst-case value. Please either quote the exact result with a hypothesis check, or prove uniqueness directly for this homogeneous quadratic example.
  4. [§IV.A.2, Lemma 13] Even after the cost function is corrected, the proof of Lemma 13 is only 'Straightforward (omitted)'. Since the lemma is load-bearing for the n>2 claim, a short proof should be included, explicitly using the decoupled dynamics and the additive structure of the cost.
minor comments (3)
  1. [§IV.A.1, Lemma 12] In the proof, the phrase '˜J ⋆ is not differentiable is not differentiable at η' contains a duplicated clause. Please fix this typo.
  2. [§IV.A.1, Corollary 4] The definition J^∘(x)=r^2 \tilde J^∘(θ) with r≥0 is ambiguous at r=0. The value at the origin should be stated explicitly (it is 0 by continuity and c(0)=0).
  3. [General] The paper cites [19, Thm 3.1] for Bellman uniqueness but does not reproduce the theorem. A precise statement in the appendix, or at least a precise reference with the assumptions, would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the construction is explicit and relies on standard external results.

full rationale

The paper's derivation chain is self-contained in the relevant sense: the value functions are defined directly from the cost-to-go sums, the Bellman equations (2)-(3) are standard consequences of the definitions, and the constructed example functions are explicit. There is no fitting of parameters to a target quantity, no renaming of an empirical pattern as a new derivation, and no load-bearing self-citation: the only cited theorems are external (Jungers, Rademacher/Rudin, Keane on interval exchange maps, Meyn's dynamic programming theorem, Niven). The appeal to [19, Thm. 3.1] in Corollary 4 is a standard verification/uniqueness result, not an import of the author's own prior work, and it is used to identify a constructed Bellman solution with the worst-case value function rather than to define it. The manuscript includes an omitted proof (Lemma 13, 'Proof: Straightforward (omitted)') and a possible correctness gap in the n>2 extension, but those are mathematical-rigor issues, not circularity: the n>2 equality is asserted as a lemma, not assumed as an input, and its failure would make the proof incorrect rather than circular. Hence no step in the claimed derivation reduces by construction to its own inputs.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

No empirically fitted parameters; all constants are explicit witnesses for an existence proof and are chosen to satisfy inequalities. The only non-standard invoked fact is Bellman uniqueness for J^circ, which is standard but not fully proved in the paper.

free parameters (3)
  • angle alpha (with beta = alpha - pi/2) = arctan(3/4) in the rational example
    Chosen irrational multiple of pi so the induced interval exchange has dense orbits; also allows A1, A2 to have rational entries.
  • contraction factor rho = 0.01
    Chosen small enough so the rho^2 J* terms in Lemma 7 cannot change the sign of Delta c; any sufficiently small rho would work.
  • cost weights = 1 and 2 (c = x1^2 + 2 x2^2)
    Chosen so c-tilde has monotonic derivative bounds in Lemmas 4 and 5; a quadratic form with unequal weights works.
assumptions (6)
  • standard math Theorem 1: if jsr < 1, there exists a norm and rho < 1 with uniform contraction
    Used in Theorem 2 and throughout; cited from [2].
  • standard math Bellman equations (Proposition 1) for optimal and worst-case value functions
    Foundation of the functional equations in Lemma 6; cited to [19].
  • domain assumption Uniqueness of continuous Bellman solutions with value 0 at the origin for stable switched systems
    Needed in Corollary 4 to identify the constructed J^circ as the worst-case value function; invoked via [19, Thm 3.1] but not proved in the paper.
  • standard math Density of orbits of irrational rotations / interval exchange maps
    Used in Lemma 9; relies on alpha/pi being irrational.
  • standard math Niven's theorem: rational values of trigonometric functions at rational multiples of pi are restricted
    Used in Lemma 14 to prove alpha/pi irrational for the rational matrices.
  • standard math Rademacher / absolute continuity: Lipschitz functions are differentiable almost everywhere
    Used in Corollary 1 to contrast with nowhere differentiable examples.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the differentiability of the value function of switched linear systems under arbitrary and controlled switching." pith.science (2026). https://pith.science/paper/5EIJUOES

@misc{pith2026251120037,
  author       = {Pith},
  title        = {Pith review of: On the differentiability of the value function of switched linear systems under arbitrary and controlled switching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5EIJUOES}},
  note         = {Machine review of arXiv:2511.20037}
}
read the original abstract

This paper studies the differentiability of the value function of switched linear systems under arbitrary switching and controlled switching, referred to as worst-case and optimal value functions respectively. First, we show that the value functions are Lipschitz continuous, when the cost function is Lipschitz continuous. Then, as the central contribution of this work, we show with examples that each of these functions can be non-differentiable on dense subsets of the state space, even if the cost function is smooth and Lipschitz continuous. This has implications for optimal control and reinforcement learning since it implies that the exact computation of these value functions requires templates involving functions that are non-differentiable on dense subsets.

Figures

Figures reproduced from arXiv: 2511.20037 by the authors.

Figure 1
Figure 1. The function c˜ and a potential plot of J˜⋆ (left y-axis), along with ∆J˜⋆ (right y-axis) for α = 0.6 and β = α − π/2 (µ is the purple line). Lemma 4: For all x ∈ [π/8, 3π/8] + πZ, c˜ ′ (x) ≥ √ 2/2, and for all x ∈ [−3π/8, −π/8] + πZ, c˜ ′ (x) ≤ −√ 2/2. Proof: Straightforward from c˜ ′ (θ) = sin(2θ). For each θ ∈ R, we let ∆˜c(θ) := ˜c(θ + α) − c˜(θ + β). We also let µ := −(α + β)/2 = π/4 − α = −π/4 − β. Lemma 5: It… view at source ↗
Figure 2
Figure 2. The truncated backward trajectory {T −1ν, . . . , T −15ν} (black dots) of the map T for α = 0.6, β = α − π/2 and ν = 0.1. The blue arrows represent transitions by T −1 . (recall that µ = π/4 − α). Hence, by Lemma 4, we get that c˜(ν + α) − c˜(θ + α) ≥ √ 2/2(ν − θ) (mean value theorem). Thus, by Lemmas 6 and 2, J˜⋆ (ν +α)−J˜⋆ (θ+α) ≥ ( √ 2/2−ρ 2M)(ν −θ) ≥ 1 2 (ν −θ), which gives h(ν) − h(θ) ≥ 1 2 (ν − θ). Proof of b)… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Robust Optimal Control of Arbitrarily Switched Systems: A Path-Complete Framework

    math.OC 2026-07 conditional novelty 6.0 of 10

    A path-complete graph framework jointly synthesizes feedback policies and certified upper bounds on the infinite-horizon value function for arbitrarily switched systems, with SDP/alternating-optimization formulations ...

Reference graph

Works this paper leans on

20 extracted references · cited by 1 Pith paper

  1. [8]

    On the value functions of the discrete- time switched LQR problem,

    W. Zhang, J. Hu, and A. Abate, “On the value functions of the discrete- time switched LQR problem,”IEEE Transactions on Automatic Con- trol, vol. 54, no. 11, pp. 2669–2674, 2009

    2009

  2. [17]

    On the continuity and smoothness of the value function in reinforcement learning and optimal control,

    H. Harder and S. Peitz, “On the continuity and smoothness of the value function in reinforcement learning and optimal control,” in2024 IEEE 63rd Conference on Decision and Control (CDC). IEEE, 2024, pp. 1935–1940

    2024

  3. [1]

    Liberzon,Switching in systems and control

    D. Liberzon,Switching in systems and control. Boston, MA: Birkh¨auser, 2003

    2003

  4. [2]

    R. M. Jungers,The joint spectral radius: theory and applications. Berlin: Springer, 2009

    2009

  5. [3]

    Sun and S

    Z. Sun and S. S. Ge,Stability theory of switched dynamical systems. London: Springer, 2011

    2011

  6. [4]

    Complexity of stability and controllability of elementary hybrid systems,

    V . D. Blondel and J. N. Tsitsiklis, “Complexity of stability and controllability of elementary hybrid systems,”Automatica, vol. 35, no. 3, pp. 479–489, 1999

    1999

  7. [5]

    Joint spectral radius and path-complete graph Lyapunov functions,

    A. A. Ahmadi, R. M. Jungers, P. A. Parrilo, and M. Roozbehani, “Joint spectral radius and path-complete graph Lyapunov functions,”SIAM Journal on Control and Optimization, vol. 52, no. 1, pp. 687–717, 2014

    2014

  8. [6]

    Stability and stabilizability of switched linear systems: a survey of recent results,

    H. Lin and P. J. Antsaklis, “Stability and stabilizability of switched linear systems: a survey of recent results,”IEEE Transactions on Automatic control, vol. 54, no. 2, pp. 308–322, 2009

    2009

Show all 20 references
  1. [7]

    On approximate dynamic programming in switching systems,

    A. Rantzer, “On approximate dynamic programming in switching systems,” inProceedings of the 44th IEEE Conference on Decision and Control. IEEE, 2005, pp. 1391–1396

    2005

  2. [9]

    Optimal control and scheduling of switched systems,

    D. G ¨orges, M. Iz ´ak, and S. Liu, “Optimal control and scheduling of switched systems,”IEEE Transactions on Automatic Control, vol. 56, no. 1, pp. 135–140, 2011

    2011

  3. [10]

    Infinite-horizon switched LQR problems in discrete time: a suboptimal algorithm with performance analysis,

    W. Zhang, J. Hu, and A. Abate, “Infinite-horizon switched LQR problems in discrete time: a suboptimal algorithm with performance analysis,”IEEE Transactions on Automatic Control, vol. 57, no. 7, pp. 1815–1821, 2012

    2012

  4. [11]

    Optimal control of hybrid switched systems: a brief survey,

    F. Zhu and P. J. Antsaklis, “Optimal control of hybrid switched systems: a brief survey,”Discrete Event Dynamic Systems, vol. 25, no. 3, pp. 345–364, 2015

    2015

  5. [12]

    Linear quadratic regulation of switched systems using informed policies,

    D. Antunes and W. M. Heemels, “Linear quadratic regulation of switched systems using informed policies,”IEEE Transactions on Automatic Control, vol. 62, no. 6, pp. 2675–2688, 2017

    2017

  6. [13]

    Linear quadratic regulator of discrete-time switched linear systems,

    G. Wu, L. Xiong, G. Wang, and J. Sun, “Linear quadratic regulator of discrete-time switched linear systems,”IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 67, no. 12, pp. 3113–3117, 2020

    2020

  7. [14]

    Optimal switching sequence for switched linear systems,

    Z. Wu and Q. He, “Optimal switching sequence for switched linear systems,”SIAM Journal on Control and Optimization, vol. 58, no. 2, pp. 1183–1206, 2020

    2020

  8. [15]

    An improved method for approximating the infinite-horizon value function of the discrete-time switched LQR problem,

    T. Hou, Y . Li, and Z. Lin, “An improved method for approximating the infinite-horizon value function of the discrete-time switched LQR problem,”Journal of Systems Science and Complexity, vol. 37, no. 1, pp. 22–39, 2024

    2024

  9. [16]

    Interval exchange transformations,

    M. Keane, “Interval exchange transformations,”Mathematische Zeitschrift, vol. 141, no. 1, pp. 25–31, 1975

    1975

  10. [18]

    Rudin,Real and complex analysis, 3rd ed

    W. Rudin,Real and complex analysis, 3rd ed. New York, NY: McGraw-Hill, 1987

    1987

  11. [19]

    Meyn,Control systems and reinforcement learning

    S. Meyn,Control systems and reinforcement learning. Padstow: Cambridge University Press, 2022

    2022

  12. [20]

    Niven,Irrational numbers

    I. Niven,Irrational numbers. Rahway, NJ: Mathematical Association of America, 1956

    1956

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.