Pith. sign in

REVIEW 4 major objections 4 minor 8 references

Interval-based validation of a nonlinear estimator

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A guaranteed maximum error for any nonlinear estimator, including a neural network, can be computed with interval arithmetic.

desk verdict A sound but thin proof-of-concept for certifying neural net estimation error with interval optimization; the guarantee is conditional on missing implementation details. read the letter →

arxiv 2411.17215 v1 pith:ROFH2FLS submitted 2024-11-26 cs.RO

Maël Godard (Lab-STICC) , Luc Jaulin (Lab-STICC) , Damien Massé (Lab-STICC) This is my paper · ORCID
classification cs.RO MSC 65G3090C2668T07
keywords intervalanalysisMoore-Skelboealgorithmnonlinearestimationneuralnetworkvalidationguaranteederrorboundglobaloptimizationbounded-errorparameter
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes that a guaranteed upper bound on the error of any nonlinear estimator can be obtained by applying the Moore-Skelboe global-optimization algorithm to the negative error function $f(x,e) = -\epsilon(x,e) = -\|x-\psi(g(x)+e)\|$ over the parameter box $X_0$ and the noise box $E$. Because the algorithm maintains an interval enclosure of the true global minimum, the quantity $-\operatorname{lb}(f(B_0))$ returned from the final box $B_0$ is a pessimistic bound that the true maximum error cannot exceed. This matters because practical estimators such as trained neural networks are not themselves guaranteed, yet the paper shows they can be certified from the outside using interval analysis. On the paper's trilateration example, the certified maximum error is $\bar{\epsilon}=1.7$, meaning every true parameter value lies within distance $1.7$ of the network's estimate throughout the tested domain.

What carries the argument

The load-bearing mechanism is the Moore-Skelboe algorithm, a global-optimization method that maintains a cover of the domain as boxes, repeatedly splits the box with the lowest interval lower bound, and reinserts the pieces sorted by that bound. Its invariant guarantees that the global minimum of the objective lies in the interval enclosure of the selected box. Applied to $f=-\epsilon$ with the initial cover $B_0=X_0\times E$, and splitting only the $X_0$ component, the returned value $-\operatorname{lb}(f(B_0))$ becomes a pessimistic certified maximum error, and the final box localizes the parameter values most sensitive to noise. The correctness rests on having a computable interval inclusion function for the full composition $\epsilon(x,e)=\|x-\psi(g(x)+e)\|$, including the relu layers of the neural network.

What would settle it

One concrete check: on a low-dimensional problem whose true maximum error is known analytically, run the method with a deliberately incorrect inclusion function (for example, treating relu as the identity) and see whether $-\operatorname{lb}(f(B_0))$ falls below the known maximum; any such under-estimate disproves the guarantee for that implementation. For the paper's reported application, a dense grid of samples from $X_0\times E$ fed through the trained network would falsify the 1.7 bound if any sample shows error above 1.7.

Watch

Extended reading notes

Core claim

The central claim is that the Moore-Skelboe algorithm, applied with objective $f=-\epsilon$ and initial cover $B_0=X_0\times E$, returns an interval $f(B_0)$ whose negative lower bound $-\operatorname{lb}(f(B_0))$ is a guaranteed maximum error for the estimator: for every $x\in X_0$ and every $e\in E$, the error $\|x-\psi(g(x)+e)\|$ is at most $-\operatorname{lb}(f(B_0))$. The proof is the standard enclosure invariant of the algorithm: the cover always covers the whole domain, so every point's value lies above $\operatorname{lb}(f(B_0))$; since the selected box is nonempty, the global minimum of $f$ lies in $f(B_0)$; translating to $-\epsilon$ turns this into an upper bound on the maximum error. The paper demonstrates the result on a relu neural-network estimator for a 2D localization problem, reporting $\bar{\epsilon}=1.7$.

Load-bearing premise

The guarantee collapses if the interval calculation for the composed error, including the trained neural network and the relu activations, does not truly enclose every value the estimator can produce, or if the arithmetic rounds inward instead of outward; in either case the computed bound can fall below the true maximum error.

Editorial extensions

If this is right

  • Any nonlinear estimator whose composed error function has a computable interval inclusion can be certified with a guaranteed maximum-error bound, even if the estimator itself is not interval-based.
  • For the paper's trilateration example, the certified bound is $\bar{\epsilon}=1.7$, so a true position is guaranteed to lie inside the radius-1.7 circle around the network's estimate.
  • Because only the parameter box is split, the final box identifies which parameter values produce the worst estimation error, giving a diagnostic for where the estimator is most fragile.
  • If the certified bound exceeds the application's tolerance, the estimator can be retrained or restructured and the certification rerun to check whether the bound improves.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the same enclosure approach could certify other black-box estimators, such as Kalman filters or learned controllers, whenever an interval inclusion of their full computation can be built.
  • Editorial extension: the reported bound of 1.7 is only as tight as the interval inclusions; using mean-value or Taylor inclusion functions would likely reduce it, making the practical value of the method depend on inclusion sharpness as much as on the splitting strategy.
  • Editorial extension: the paper's 'split only the parameter box' design produces a sensitivity map; this could be turned into an active-learning criterion that retrains the estimator selectively in the worst parameter regions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes an interval-based method for certifying the maximum estimation error of a nonlinear estimator. The error function ε(x,e)=||x−ψ(g(x)+e)|| is maximized by applying the Moore–Skelboe global optimization algorithm to f=−ε over the box X0×E; the returned value −lb(f(B0)) is argued to be a guaranteed upper bound on the true maximum error. The method is demonstrated on a neural-network estimator for a 2-D trilateration problem, reporting a bound of ¯ε=1.7. The theoretical argument in Sections 2.4 and 3 is elementary and correct under the standard assumption that a valid interval inclusion function for the composed error is available, but the paper does not provide or construct that inclusion function for the neural network, and the numerical result is not accompanied by enough implementation detail to be verified.

Significance. If the missing inclusion-function machinery is supplied, the method is a simple and potentially useful way to certify estimators that are not themselves interval-based. The core idea—negating the error function and taking the negative lower bound returned by Moore–Skelboe—is sound and clearly stated, and the proof of the bound from the covering property is correct. The application to a neural network is an appealing demonstration of certifying a non-guaranteed estimator. The paper is not fully reproducible as written: the trained network weights, the interval implementation, the outward-rounding behavior, and the stopping criterion are all absent. The central theorem is conditional on an assumption that is stated but neither proven nor instantiated, which currently limits the paper's contribution to a conditional proof rather than an demonstrated validated estimator.

major comments (4)
  1. [§2.2] The statement 'For any function f : Rn → Rm, its definition can be extended to IRn and we also call f : IRn → IRm its natural inclusion function' is not true for arbitrary functions and is the load-bearing assumption of the paper. The guarantee of Section 3 requires a valid interval inclusion function for ε(x,e)=||x−ψ(g(x)+e)||, including the Euclidean norm and the trained neural network with relu activations. No construction, proof, or condition for the existence of such an inclusion function is given. The authors should either state precise conditions under which a natural inclusion function exists or explicitly construct an inclusion function for the composed error, including how the neural network weights and biases are enclosed.
  2. [§4.3] The reported value ¯ε=1.7 is not verifiable from the manuscript. The trained neural network is only described by its layer sizes and activations; the actual weights and biases are not given, no interval arithmetic library or outward-rounding implementation is identified, and the stopping criterion δ and the number of Moore–Skelboe iterations are not reported. Because the bound is specific to the trained network, these details are necessary to check that the computed value truly encloses the maximum error. The paper should include the trained network (e.g., as supplementary weights or a code repository) and the complete evaluation settings, or clearly state that the numerical value is an illustration rather than a reproducible guarantee.
  3. [§3] The sentence 'Upon splitting B0, only its component belonging to X0 will be splitted' is not justified. If the noise box E is never split, then the interval evaluation of f over E is never refined, so the width w(f(B0)) may remain large regardless of the refinement in the parameter directions. This could prevent the Moore–Skelboe stopping criterion w(f(B0)) ≤ δ from being satisfied, or it may produce an arbitrarily pessimistic bound. The authors should clarify whether the method intentionally returns a conservative bound without expecting convergence, or explain why splitting only X0 is sufficient for the algorithm's termination and for recovering the maximum over both X0 and E.
  4. [§4.2] The neural network description is incomplete for the purpose of interval evaluation. In particular, the output layer applies a relu activation to the two-dimensional output, which restricts estimates to the nonnegative orthant; this is consistent with X0=[5,25]^2 but should be stated explicitly because it affects the interval inclusion function. More importantly, the trained weights are absent, so the interval evaluation of the network is not defined. This is a specific instance of the missing inclusion-function machinery, but it merits separate attention because the architecture choice (relu on the output) has a direct impact on the error function's range.
minor comments (4)
  1. [§1] The introduction states that Section 4 presents 'the validation of a gradient descent based estimator and a neural network estimator', but the application section only describes the neural network estimator. The gradient-descent estimator is never defined or validated elsewhere in the manuscript.
  2. [References] The reference to Gamelin and Greene is incorrectly titled: the entry reads 'Evaluating generalization through interval-based neural network inversion' but the actual book is 'Introduction to Topology, Second Edition'. The title appears to have been copied from the preceding entry.
  3. [§2.2] The definition of a natural inclusion function is informal; the paper would benefit from stating the inclusion property f([x]) ⊇ {f(x) : x∈[x]} explicitly, since the proof in Section 2.4 relies on it.
  4. [§2.4] The proof of inequality (8) uses that lb(f(B0)) is the smallest lower bound among all boxes in the cover, but the notation is slightly ambiguous because B0 is reused as the box selected in the while loop and as the entire initial cover. Clarifying the index of the selected box would improve readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the error bound follows directly from the estimator's defining equations, and the only self-citation is non-essential background.

full rationale

The derivation chain is: define the estimation error in Eq. (3) as epsilon(x,e)=||x - psi(g(x)+e)||, set the target as its maximum over X0 x E in Eqs. (4)-(6), and reformulate it as the minimization of f = -epsilon. Section 3 applies Moore-Skelboe to f with initial box B0 = X0 x E, and Eq. (11) gives the invariant that the global minimum lies in the interval f(B0); therefore -lb(f(B0)) is a guaranteed upper bound on the maximum of epsilon. This is a direct computation from the estimator psi and the given parameter/noise boxes, with no parameter fitted to make the bound hold. The neural-network application in Section 4 replaces psi by the trained network; that network is the object under test, not an input to the method, so the reported bound is not a fitted constant disguised as a prediction. The only self-citation is Jaulin and Walter (1993) in Section 1, used as background for interval-analysis estimators; it does not supply any step of the Moore-Skelboe argument, so it is non-load-bearing. There is a reproducibility limitation worth noting: Section 2.2 asserts without construction that every function has a natural interval inclusion function, and Section 4.3 reports bar-epsilon = 1.7 without giving the network weights, interval library, outward-rounding behavior, stopping criterion, or number of iterations. That gap affects verification of the numerical value but is not a circularity, since the bound is not built from itself or from a fitted constant.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central method adds no fitted constants; the bound is derived from interval enclosures. However, the numerical example depends on an unpublished trained neural network and unstated algorithm parameters, and the guarantee relies on the existence and correct implementation of inclusion functions for the error function.

free parameters (3)
  • Trained neural network weights = not provided
    The bound bar-epsilon = 1.7 is specific to a network trained on noisy data; without the weights the numerical claim cannot be checked.
  • Stopping criterion delta for Moore-Skelboe = not stated
    Algorithm 1 requires delta; the paper never gives its value, although the pessimistic bound remains valid for any delta.
  • Neural network training hyperparameters (optimizer, epochs, loss, dataset size) = not stated
    These determine the trained estimator and therefore the computed error bound; their absence blocks reproduction.
assumptions (5)
  • domain assumption A computable inclusion function exists for f = -epsilon on X0 x E.
    Section 3 applies Moore-Skelboe to f without proving that an interval extension of the trained neural network and the Euclidean norm is available.
  • standard math Moore-Skelboe with inclusion functions encloses the global minimum.
    Correctness is sketched in Section 2.4 via lb(f(B0)) <= mu <= rb(f(B0)) and delegated to van Emden 2004.
  • domain assumption Interval arithmetic is implemented with outward rounding.
    The paper labels the result guaranteed but does not state the interval library or rounding mode.
  • domain assumption The neural network is deterministic and fixed after training.
    The bound applies to one specific trained network; weights are not provided so the analyzed and deployed networks cannot be compared.
  • domain assumption The noise model E covers all possible disturbances.
    The maximum is taken only over X0 x E in Section 3; any noise outside E invalidates the guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interval-based validation of a nonlinear estimator." pith.science (2026). https://pith.science/paper/ROFH2FLS

@misc{pith2026241117215,
  author       = {Pith},
  title        = {Pith review of: Interval-based validation of a nonlinear estimator},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ROFH2FLS}},
  note         = {Machine review of arXiv:2411.17215}
}
read the original abstract

In engineering, models are often used to represent the behavior of a system. Estimators are then needed to approximate the values of the model's parameters based on observations. This approximation implies a difference between the values predicted by the model and the observations that have been made. It creates an uncertainty that can lead to dangerous decision making. Interval analysis tools can be used to guarantee some properties of an estimator, even when the estimator itself doesn't rely on interval analysis (Adam, 2019) (Adam, 2015). This paper contributes to this dynamic by proposing an interval-based and guaranteed method to validate a nonlinear estimator. It is based on the Moore-Skelboe algorithm (van Emden, 2004). This method returns a guaranteed maximum error that the estimator will never exceed. We will show that we can guarantee properties even when working with non-guaranteed estimators such as neural networks.

Figures

Figures reproduced from arXiv: 2411.17215 by the authors.

Figure 1
Figure 1. Estimation problem To validate an estimator we want to evaluate the maximum of its error function on X0 × E, noted ¯ϵ. The problem can written as below: ϵ¯ = max ϵ(x, e) (4) x ∈ X0 (5) e ∈ E (6) 2.2. Intervals and boxes Intervals are used to represent a range of value instead of a single one. An interval can be defined by two of the following: − Its lower bound, or left bound, noted lb − Its upper bound, or right bo… view at source ↗
Figure 2
Figure 2. Moore-Skelboe algorithm example 3. Problem Formulation and method To validate an estimator, we want to estimate the maximum error that it commits on a given set of possible parameters. This maximisation problem on the function ϵ can also be interpreted as a Optimization problem on the function −ϵ. The Moore-Skelboe algorithm can then be applied with the objective function f = −ϵ and with {B0 = X0 × E} as the initial… view at source ↗
Figure 3
Figure 3. Application example The norm used below to calculate the distances is the Euclidian norm. For numerical applications, we will take : − X0 = [5, 25]2 − E = [−0.2, 0.2]3 − a = (10, −9) ; b = (5, 12) ; c = (−15, 0) 4.2. Neural network estimator Neural networks are a common example of non-guaranteed estimator. To prove the efficiency of our method, we created a neural network estimator represented [PITH_FULL_IMAGE:figu… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Neural network estimator This neural network has then been trained on noisy data. The estimation is finally obtained by making a prediction with the resulting trained neural network. 4.3. Numerical application The maximum error given by the Moore-Skelboe algorithm in t…
Figure 5
Figure 5. Figure 5: Simulation to visualise ¯ϵ Depending on the accuracy required by the application, we can then either use the neural network as it is or train it again, possibly by modifying its structure. Using this method, we were able to find a guaranteed property for a neural netwo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 8 canonical work pages

  1. [1]

    Mahaboob, M

    Lakshmi, K., B. Mahaboob, M. Rajaiah, and C. Narayana. Ordinary least squares estimation of parameters of linear model . Journal of Mathematical and Computational Science , 2021

    work page 2021

  2. [2]

    Nonlinear Least Squares for Inverse Problems: Theoretical Foundations and Step-by-Step Guide for Applications

    Chavent, G. Nonlinear Least Squares for Inverse Problems: Theoretical Foundations and Step-by-Step Guide for Applications . Springer , 2010

    work page 2010

  3. [3]

    Hartley, H. O. and A. Booker. Nonlinear least squares estimation . The Annals of Mathematical Statistics , 1965

    work page 1965

  4. [4]

    Jaulin, L. and E. Walter. Set inversion via interval analysis for nonlinear bounded-error estimation . Automatica , 29(4):1053--1064, 1993

    work page 1993

  5. [5]

    Adam, S. P., D. A. Karras, G. D. Magoulas, and M. N. Vrahatis. Reliable estimation of a neural network’s domain of validity through interval analysis based inversion . In 2015 International Joint Conference on Neural Networks (IJCNN) , 1--8, 2015

    work page 2015

  6. [6]

    Adam, S.P., A. C. Likas and M. N Vrahatis. Evaluating generalization through interval-based neural network inversion . Neural Comput and Applic , 31:9241--9260, 2019

    work page 2019

  7. [7]

    W., and R

    Gamelin, T. W., and R. E. Greene. Evaluating generalization through interval-based neural network inversion . In Dover Publications, Introduction to Topology, Second Edition , 1999

    work page 1999

  8. [8]

    van Emden, M. and B. Moa. Termination criteria in the Moore-Skelboe Algorithm for Global Optimization by Interval Arithmetic . In Floudas, C.A., Pardalos, P. (eds) Frontiers in Global Optimization. Nonconvex Optimization and Its Applications , vol 74. Springer, Boston, MA, 2004

    work page 2004

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.