{"id":"25d5a984-4ba1-4f3a-9805-cc3a07c7044d","arxiv_id":"2506.23977","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A loop-transformation convexification plus a randomized subspace sketch for Lipschitz-constrained training; the sketch's high-probability certificate is not mathematically justified.","lead":"This paper proposes a training method that aims to give neural networks a hard upper bound on how much their output can change when inputs are perturbed. The fast, scalable version of the method is shown to rely on a probabilistic shortcut that does not actually prove the bound it claims.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The sketched LMI in RS-LMI is necessary but not sufficient for the per-layer spectral-norm certificate; a fixed Gaussian sketch can certify ||W_k||_2 <= sqrt(tau_k) with non-negligible probability even when it is false, so the claimed global Lipschitz guarantee does not hold.","rationale":"The reader's weakest assumption—that a fixed Gaussian sketch preserves PSD feasibility—is the correct load-bearing point. The cited Johnson-Lindenstrauss lemma only controls distances between a finite set of vectors; the PSD cone is not a finite set, and a random subspace need not intersect the negative-curvature cone. The explicit 2x2 example shows non-negligible failure probability even for m = 1; for larger m the probability can be smaller but remains positive and unquantified, and the sketch is fixed at initialization with no verification. Because the conclusion '||W_k||_2 <= sqrt(tau_k) with high probability' is used to certify the product bound and to claim guaranteed robustness, the main scalable contribution fails as stated. I also note that the asserted equality in Theorem 5.1 between the global LipSDP bound and the product of spectral norms is not correct in general, but the sketch-sufficiency issue is sufficient to reject. I agree with the reader's REJECT verdict: a revised paper would need either a deterministic certification check after training, or a rigorous probabilistic guarantee with a quantifiable failure probability and a per-layer method to detect when the sketch fails. The paper may still be useful as a heuristic or as a cautionary example, but the stated safety guarantee is not supported.","tokens_in":10668,"tokens_out":4516,"duration_ms":51561,"concrete_test":"Reproduce the single-layer counterexample: fix W = diag(1.2, 1.0), tau = 1.1, m = 1. Draw 10^5 Gaussian vectors g in R^2 and compute g^T(tau I - W^T W)g. If any sample yields a nonnegative value (or if the sketched penalty can be driven to zero at tau < 1.44), then the sketched condition is satisfiable for a matrix whose spectral norm exceeds sqrt(tau), directly falsifying Remark 3. A stronger end-to-end test: after running RS-LMI on MNIST or CIFAR-10, compute the exact spectral norm of every trained weight matrix and compare with sqrt(tau_k); a single violation ||W_k||_2 > sqrt(tau_k) invalidates the claimed certificate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RS-LMI provides a valid global Lipschitz certificate (Remark 3 and the abstract). This rests on the assertion that a fixed Gaussian sketch G_k^T(tau_k I - W_k^T W_k) G_k ⪰ 0 implies tau_k I - W_k^T W_k ⪰ 0 with high probability, invoked after Eq. (25) as a Johnson-Lindenstrauss guarantee. That implication is false. JL preserves distances between finitely many vectors; it says nothing about membership in the PSD cone, which is an infinite-dimensional set. For a fixed indefinite matrix A = tau_k I - W_k^T W_k, a random m-dimensional subspace can miss the negative-curvature directions entirely, so the sketched condition can hold while A has negative eigenvalues. Concretely, for W = diag(1.2, 1.0) and tau = 1.1, A = diag(-0.34, 0.1). For m = 1, a Gaussian sketch g satisfies g^T A g >= 0 with probability about 2*arctan(sqrt(0.1/0.34))/pi ≈ 0.16, certifying sigma_max(W) <= 1.049 while the true value is 1.2. The failure probability depends on W and is not vanishing; there is no deterministic check that the sketch is 'good.' Thus the converged product over layers of sqrt(tau_k) is not a valid certificate, and the advertised robustness guarantee is unsupported. A secondary issue is that Theorem 5.1 asserts equality between the global LipSDP bound and the product of per-layer spectral norms, but that product is only an upper bound on the Lipschitz constant in general. The sketch-sufficiency error remains the load-bearing one because it invalidates the scalable contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a framework for training neural networks under global Lipschitz constraints. It first introduces Lip-Loop, a loop-transformation reparameterization intended to make the LipSDP admissibility condition convex in the decision variables, and then RS-LMI, a randomized sketching scheme that replaces per-layer LMI constraints with low-dimensional sketched constraints to reduce computational cost. The authors claim that RS-LMI provides a valid global Lipschitz certificate with high probability and demonstrate improved runtime and memory usage on MNIST, CIFAR-10, and ImageNet. The central theoretical support for the certificate claim is that a Gaussian sketch preserves the feasibility of the per-layer LMI, and that the global Lipschitz bound equals the product of per-layer spectral norms.","tokens_in":10993,"tokens_out":4390,"duration_ms":49895,"significance":"If the main claims were correct, the paper would offer a practically important advance: a scalable, certifiable Lipschitz-constrained training method with convex structure and empirical gains over existing SDP-based approaches. The empirical comparisons in Table I and Table II are suggestive, and the paper makes useful connections between loop transformation and neural network Lipschitz analysis. However, the load-bearing theoretical claims are not supported. The randomized sketching step does not preserve the PSD feasibility of the LMI, so the claimed valid global certificate does not follow; the equality in Theorem 5.1 between the global LipSDP constant and the product of layerwise spectral norms is not generally true; and the overall training problem remains nonconvex despite the 'convex' framing. These issues undermine the paper's core novelty, namely the certifiable scalability result. The paper does not provide machine-checked proofs or released code, and the experimental section lacks the detail needed to verify the efficiency claims independently.","major_comments":[{"comment":"The claim that drawing a fixed Gaussian sketch G_k and imposing G_k^T(τ_k I - W_k^T W_k)G_k ⪰ 0 preserves the Lipschitz certificate with high probability is not justified and is false. The Johnson-Lindenstrauss lemma controls distances between finitely many points; it does not preserve membership in the positive semidefinite cone. The sketched inequality is necessary for τ_k I - W_k^T W_k ⪰ 0, but not sufficient: for W = diag(1.2, 1.0), τ = 1.1, the matrix A = diag(-0.34, 0.1) is indefinite, yet a random one-dimensional Gaussian sketch satisfies g^T A g ≥ 0 with probability about 0.16. Thus a converged RS-LMI solution can certify a false bound ∥W_k∥2 ≤ √τ_k, and the product over layers in Remark 3 is not a valid global certificate. This invalidates the central scalability claim of the paper.","section":"Section V, Eq. (25) and Remark 3"},{"comment":"Theorem 5.1 asserts equality between the global LipSDP Lipschitz constant and the product of per-layer spectral norms. Only the inequality L ≤ ∏ σ_max(W_k) holds in general for slope-restricted activations; equality fails even for simple ReLU networks (e.g., a two-layer abs-like network has Lipschitz constant 1 while the product of spectral norms is 2). The theorem provides no proof of equality and the condition 'each activation is 1-Lipschitz' does not imply equality. Consequently the claim that the global LipSDP 'can split into independent layer-wise SDPs' is not established, and RS-LMI's reported bounds should not be described as recovering the Lip-Loop bound.","section":"Section V, Theorem 5.1"},{"comment":"The paper calls the training framework 'convex' and states that any converged solution provides safety guarantees, but optimization problem (20) contains the nonconvex equality constraint f(N)Q = K together with L(N) as a nonlinear function of N. The ADMM updates (22)-(24) are therefore not guaranteed to converge to a feasible point, and no convergence analysis is provided. The statement that the N-update 'typically converges to local optima' and the (L,Q,K)-update obtains a global optimum does not imply convergence of the alternating scheme to a point satisfying f(N)Q = K. Hence the 'convex admissibility condition' is only a convexification of part of the problem and the overall training algorithm lacks the advertised guarantee.","section":"Section IV-C, Eq. (20)"}],"minor_comments":[{"comment":"There are numerous typos and grammatical errors, including 'A principle approach' in the abstract, 'yielf' in the introduction, 'access scalability' in Section VI, and 'Imagnet' in Table II discussion; these should be corrected.","section":"Throughout"},{"comment":"The experimental section omits key details needed for reproducibility: the sketch dimension m, the penalty weights α_k, the ADMM penalty ρ, the optimizer and hyperparameters, the exact network architectures, and the hardware/software environment. Without these, the reported runtime and memory comparisons cannot be independently verified.","section":"Section VI"},{"comment":"The loop transformation derivation is deferred to the authors' previous work [20] with no summary of the assumptions or proof, even though this is the source of the convex reparameterization and a central contribution of the paper; a self-contained statement of the transformation and its validity would strengthen the manuscript.","section":"Section IV-A"},{"comment":"The sentence 'Training time is reduced by over 90% on MNIST by GloRo and over 80% on CIFAR-10 by LipDiff' is confusing; it should specify the comparison baseline and dataset for each percentage.","section":"Table II"},{"comment":"Reference [28] is cited for the Johnson-Lindenstrauss lemma, but that reference is about randomized sketches of convex programs; a direct citation to a Johnson-Lindenstrauss statement would be more appropriate, and the linkage should be made explicit.","section":"References"}],"recommendation":"reject","confidential_remarks":"The main theoretical contribution, the RS-LMI certificate guarantee, is invalid as stated, and the equality claim in Theorem 5.1 is not generally true; these are load-bearing errors rather than presentation issues. The empirical results may still be of interest as heuristics, but the manuscript as written does not establish the advertised robustness guarantees."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's main theoretical contribution is not sound. The randomized sketching step, which is the entire basis for the scalability claim, preserves only a necessary condition, not a certificate. The paper would need a very different argument to support the stated guarantee.\n\nWhat's genuinely useful here: the experiments show that a sketched eigenvalue penalty can train networks with competitive accuracy and fast runtime. That is a plausible heuristic, and the ImageNet results are new. The Lip-Loop convex reparameterization is also clearly explained, though the authors attribute it to their own prior work, so it is not a new contribution.\n\nThe problems are load-bearing. The claim after Eq. (25) that a fixed Gaussian sketch preserves the PSD feasibility of the LMI \"with high probability\" via Johnson-Lindenstrauss is a misapplication. JL preserves distances between finitely many points; it says nothing about cone membership. The sketched condition G^T(tau I - W^T W)G >= 0 is necessary for tau I - W^T W >= 0, not sufficient. A small indefinite matrix can easily have most random directions with nonnegative quadratic form; the concrete counterexample in the stress test (W=diag(1.2,1.0), tau=1.1, m=1) gives a certification probability around 0.16 for a false bound. The failure probability depends on the weights and does not vanish. So the \"valid global certificate\" in Remark 3 is not valid.\n\nTheorem 5.1 also overstates: the global LipSDP bound is not generally equal to the product of per-layer spectral norms; that product is only a loose upper bound. The decomposition misses inter-layer coupling, so the equality claim is false.\n\nThe empirical comparisons lack error bars and code, which is a concern for a paper whose main quantitative claims are about speed and bound tightness. The ADMM convergence limitation is honestly acknowledged, but the statement that \"any converged solution provides safety guarantees\" only holds if the constraints are exactly enforced, which is not the case in RS-LMI.\n\nBottom line: this is a reasonable heuristic for fast Lipschitz regularization, not a certified robustness method. I would not invite a full review of the current version; the main theorem fails. The authors need to either provide a sufficient condition for sketch preservation (unlikely with fixed sketches) or reframe the work as a heuristic with clear caveats and public code. As submitted, I would desk-reject.","headline":"The RS-LMI sketch does not certify a Lipschitz bound, so the paper's central claim fails, but the empirical heuristic may have value.","tokens_in":11599,"tokens_out":3750,"would_cite":false,"duration_ms":41298,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C22","68T07","90C25"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that Lipschitz-constrained training can be made convex and scalable by reparameterizing the network with a loop transformation and by replacing the global SDP with randomized subspace LMIs, yielding certified robustness…","keywords":["Lipschitz-constrained training","certified robustness","semidefinite programming","loop transformation","randomized sketching","linear matrix inequalities","adversarial robustness","neural network verification"],"falsifier":"Take any trained RS-LMI network, compute the true spectral norm $\\|W_k\\|_2$ of every layer, and compare it with $\\sqrt{\\tau_k}$ from the converged penalty; if any layer has $\\|W_k\\|_2 > \\sqrt{\\tau_k}$ outside solver tolerance, the Remark 3 certificate fails. A reader can also generate random matrices $W$, draw a Gaussian $G$, and check whether $G^\\top(\\tau I - W^\\top W)G \\succeq 0$ can hold while $\\tau < \\|W\\|_2^2$; one such example would falsify the claimed implication.","tokens_in":10373,"feed_emoji":"🛡️","tokens_out":12668,"duration_ms":116247,"temperature":0.7,"pith_summary":"The paper sets out to make certified robust training of neural networks tractable at scale. It proposes Lip-Loop, a convex reparameterization of the standard LipSDP condition via loop transformation, so that a semidefinite admissibility constraint can be imposed while training rather than after. Because the resulting global SDP is expensive, it also proposes RS-LMI, which splits the certificate into per-layer linear matrix inequalities and projects each onto a random low-dimensional subspace, yielding a smooth differentiable penalty. The paper reports that on MNIST, CIFAR-10, and ImageNet these methods match or beat baseline accuracy while producing tighter Lipschitz bounds and cutting runtime and memory by an order of magnitude or more. If the certificate claim holds, this would give a practical route to neural networks with formal robustness guarantees.","feed_headline":"Convex training certifies Lipschitz bounds at a fraction of the cost","feed_subtitle":"Lip-Loop and RS-LMI deliver tighter robustness bounds than spectral-norm methods and scale to ImageNet","key_machinery":"The load-bearing objects are two. First, the loop transformation: a linear fractional reparameterization $\\tilde{N} = f(N)$ that shifts each slope-restricted activation into the sector $[-1,1]$, turning the bilinear LipSDP inequality into the convex LMI (18) in the paper, linear in the transformed variables. Second, the randomized subspace LMI (RS-LMI): each per-layer PSD constraint $\\tau_k I - W_k^\\top W_k \\succeq 0$ is replaced by its projection $G_k^\\top(\\tau_k I - W_k^\\top W_k)G_k \\succeq 0$ onto a fixed Gaussian sketch $G_k \\in \\mathbb{R}^{n_{k-1}\\times m}$, and the sketched condition becomes the differentiable penalty $P_k(W_k,\\tau_k) = \\|[G_k^\\top W_k^\\top W_k G_k - \\tau_k I]_+\\|_F^2$. The claimed effect is to reduce the per-batch complexity from $O((\\sum_k n_k)^3)$ to $\\sum_k O(n_k m^2 + m^3)$ while preserving the certificate with high probability.","core_discovery":"On the paper's own terms, the central discovery is that the nonconvex LipSDP admissibility condition becomes a convex linear matrix inequality after a loop transformation that normalizes every activation to the sector $[-\\mathbf{1},\\mathbf{1}]$. Writing the transformed network as $\\tilde{N}$ and setting $Q_1 = T^{-1}$, the authors derive the LMI $\\mathrm{LMI}(Q,L,K) \\succeq 0$ that is jointly convex in $(Q_1,K_1,K_2,K_3,K_4)$, so the network map, the multiplier, and the Lipschitz bound can be optimized together via ADMM; any converged point yields a network with the certified bound. For scalability, Theorem 5.1 splits the global certificate into independent per-layer conditions $\\tau_k I - W_k^\\top W_k \\succeq 0$, whose tight solution is $\\tau_k^\\star = \\sigma_{\\max}(W_k)^2$, and RS-LMI replaces each by a Gaussian sketch $G_k^\\top(\\tau_k I - W_k^\\top W_k)G_k \\succeq 0$ encoded as a positive-semidefinite-cone penalty. The paper claims that at convergence $\\|W_k\\|_2 \\le \\sqrt{\\tau_k}$ with high probability and that $\\prod_{k=1}^l \\sqrt{\\tau_k}$ is therefore a valid global certificate.","pith_inferences":["Editorial inference: the certificate in Remark 3 is not automatic from a Johnson–Lindenstrauss guarantee, because JL preserves distances between finitely many vectors, not membership in the positive semidefinite cone; a rigorous version would need a sketch that preserves PSD feasibility with an explicit failure probability, or a post-hoc check of the full $\\tau_k I - W_k^\\top W_k$.","Editorial inference: the per-layer split in Theorem 5.1 assumes 1-Lipschitz activations; extending the loop-transformation idea to general slope-restricted activations inside the RS-LMI penalty could yield tighter bounds for tanh or leaky ReLU without losing convexity.","Editorial inference: one testable extension is to make the sketch adaptive or data-dependent and measure, on random weights, how often the sketched penalty is active while the true spectral norm exceeds $\\sqrt{\\tau_k}$; that frequency is an empirical upper bound on the failure probability of the certificate.","Editorial inference: since the loop transformation is a standard linear-fractional tool, the same convexification strategy could be carried over to other safety constraints expressible as IQCs, such as stability or passivity margins in learned controllers."],"forward_implications":["If the central claim is correct, a trained network's global Lipschitz bound can be read off as $\\prod_k \\sqrt{\\tau_k}$ without solving a global SDP, making certified training feasible for architectures as large as ResNet-18 on ImageNet.","Lip-Loop gives a convex admissibility condition that can be enforced during training, so Lipschitz constraints no longer need to be imposed after the fact by spectral-norm clipping or by fixing the SDP multiplier $T$ conservatively.","The per-layer decomposition implies that certification time and memory scale with the largest layer rather than the whole network, which is what the reported 10–50$\\times$ speedups and memory reductions rest on.","Because any converged ADMM point satisfies the LMI, the framework can in principle be combined with any differentiable loss, including losses for fairness or reinforcement-learning objectives, as the authors note."],"supporting_citations":[{"why":"Supplies the LipSDP semidefinite admissibility condition that Lip-Loop convexifies via loop transformation.","marker":"[1]"},{"why":"Provides the prior SDP-based training method and the counterexample issue that motivates the diagonal multiplier and convex reparameterization.","marker":"[11]"},{"why":"The authors' earlier loop-transformation derivation for neural networks under input-output specifications, on which the reparameterization is built.","marker":"[20]"},{"why":"Provides the incremental quadratic constraint lemma for slope-restricted activations used in the admissibility analysis.","marker":"[26]"},{"why":"Introduces loop transformation for stability and safety guarantees in control, the tool that makes the LMI convex.","marker":"[19]"},{"why":"Cited as the Johnson-Lindenstrauss-type guarantee behind RS-LMI's claim that sketching preserves the per-layer LMI with high probability.","marker":"[28]"},{"why":"Eigenvalue-based differentiable SDP baseline whose runtime and memory RS-LMI is compared against on CIFAR-10 and ImageNet.","marker":"[24]"},{"why":"Provides the barrier-method bilinear SDP training baseline and illustrates the scalability issue the paper targets.","marker":"[21]"}],"fun_headline_variants":["Convex certifiable Lipschitz training scales to ImageNet","Loop transformation makes Lipschitz training convex and fast","RS-LMI certifies Lipschitz bounds at ImageNet scale","Convex LMI formulation slashes Lipschitz certification cost","Certified robustness via convex sketching scales to full ImageNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a fixed random projection of each layer's constraint preserves the constraint's feasibility: if the projected matrix has only nonnegative eigenvalues, then the full matrix $\\tau_k I - W_k^\\top W_k$ also has only nonnegative eigenvalues.","fun_headline_variants_meta":{"raw":{"variants":["Convex certifiable Lipschitz training scales to ImageNet","Loop transformation makes Lipschitz training convex and fast","RS-LMI certifies Lipschitz bounds at ImageNet scale","Convex LMI formulation slashes Lipschitz certification cost","Certified robustness via convex sketching scales to full ImageNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000454,"raw_usage":{"total_tokens":2332,"prompt_tokens":1043,"completion_tokens":1289,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":1205}},"tokens_in":659,"tokens_out":1289,"duration_ms":10170,"temperature":1.0,"reasoning_tokens":1205,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:26:31.182120+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take any trained RS-LMI network, compute the true spectral norm $\\|W_k\\|_2$ of every layer, and compare it with $\\sqrt{\\tau_k}$ from the converged penalty; if any layer has $\\|W_k\\|_2 > \\sqrt{\\tau_k}$ outside solver tolerance, the Remark 3 certificate fails. A reader can also generate random matrices $W$, draw a Gaussian $G$, and check whether $G^\\top(\\tau I - W^\\top W)G \\succeq 0$ can hold while $\\tau < \\|W\\|_2^2$; one such example would falsify the claimed implication.","supporting_citations":[{"cited_title":"Efficient and accurate estimation of lipschitz constants for deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the LipSDP semidefinite admissibility condition that Lip-Loop convexifies via loop transformation."},{"cited_title":"Training robust neural networks using lipschitz bounds,","cited_arxiv_id":null,"evidence_quote":"Provides the prior SDP-based training method and the counterexample issue that motivates the diagonal multiplier and convex reparameterization."},{"cited_title":"Learning neural networks under input-output specifications,","cited_arxiv_id":null,"evidence_quote":"The authors' earlier loop-transformation derivation for neural networks under input-output specifications, on which the reparameterization is built."},{"cited_title":"Safety verification and robustness analysis of neural networks via quadratic constraints and semidefinite programming,","cited_arxiv_id":null,"evidence_quote":"Provides the incremental quadratic constraint lemma for slope-restricted activations used in the admissibility analysis."},{"cited_title":"Imitation learning with stability and safety guarantees,","cited_arxiv_id":null,"evidence_quote":"Introduces loop transformation for stability and safety guarantees in control, the tool that makes the LMI convex."},{"cited_title":"Randomized sketches of convex programs with sharp guarantees,","cited_arxiv_id":null,"evidence_quote":"Cited as the Johnson-Lindenstrauss-type guarantee behind RS-LMI's claim that sketching preserves the per-layer LMI with high probability."},{"cited_title":"On the scalability and memory efficiency of semidefinite programs for lipschitz constant estimation of neural networks,","cited_arxiv_id":null,"evidence_quote":"Eigenvalue-based differentiable SDP baseline whose runtime and memory RS-LMI is compared against on CIFAR-10 and ImageNet."},{"cited_title":"Neural network training under semidefinite constraints,","cited_arxiv_id":null,"evidence_quote":"Provides the barrier-method bilinear SDP training baseline and illustrates the scalability issue the paper targets."}],"review_version":1}