{"id":"d8b0689b-855d-403f-8230-1537a87be919","arxiv_id":"2605.24477","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":8.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Establishes measure-theoretic foundations for NML in regular non-smooth models and introduces the PDL-PPMH geometric MCMC sampler to compute stochastic complexity exactly.","lead":"This paper develops a geometric measure theory framework to define and compute the Normalized Maximum Likelihood codelength for non-smooth estimators such as Lasso. It introduces a specialized MCMC sampler and shows the exact NML can match cross-validation performance without data splitting.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Coarea formula extension via set-valued conservative Jacobians for PDL models lacks explicit measure-theoretic construction for NML well-posedness.","rationale":"The reader's weakest_assumption isolates exactly the regularity needed for coarea + conservative Jacobians; the load-bearing risk is whether that regularity suffices for a unique measure when the Jacobian is set-valued. Agreement is therefore full. The sampler and high-dimensional Lasso experiment address computation rather than the foundational measure-theoretic step, so they do not mitigate this particular concern.","tokens_in":1740,"tokens_out":340,"duration_ms":27699,"concrete_test":"Take the 1D soft-thresholding operator (scalar Lasso) as a PDL estimator. Compute the NML integral once via the coarea formula applied to a conservative Jacobian selection and once via direct Lebesgue integration over the parameter; if the two numerical values differ by more than machine precision, the extension does not preserve the claimed measure.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that the coarea formula, classically stated for Lipschitz maps with approximate differentials, extends to the set-valued conservative Jacobians arising in path-differentiable Lipschitz (PDL) estimators. The paper asserts this extension yields a well-posed stochastic complexity consistent with automatic differentiation outputs. The least secure step is the passage from a selection of the conservative Jacobian (a measurable selection) to a unique measure on the level sets; different selections could in principle produce different Hausdorff measures, and no explicit invariance argument or selection theorem is indicated as securing independence. This directly underpins both the well-posedness proof and the claimed consistency with AD.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper develops a measure-theoretic framework extending the coarea formula to conservative Jacobians of path-differentiable Lipschitz (PDL) estimators, proving that the Normalized Maximum Likelihood (NML) stochastic complexity is well-posed for regular non-smooth models and consistent with automatic differentiation outputs. It introduces the PDL-PPMH geometric MCMC sampler with stochastic tangent proposals and a convergent non-smooth projection solver, and demonstrates exact NML computation on high-dimensional Lasso (P=2000) as a data-efficient alternative to cross-validation.","tokens_in":1894,"tokens_out":451,"duration_ms":33020,"significance":"If the central extension holds, the result would enable exact NML computation for non-smooth estimators common in modern ML (Lasso, sparse SVMs), providing a theoretically grounded alternative to cross-validation without data splitting. The geometric sampler and its justification constitute a technical contribution to sampling on non-differentiable level sets.","major_comments":[{"comment":"The extension of the coarea formula to set-valued conservative Jacobians for PDL models asserts a well-posed NML via a unique Hausdorff measure on level sets, but provides no explicit invariance argument or selection theorem showing independence from the choice of measurable selection of the Jacobian. Different selections could yield different measures, directly undermining the well-posedness claim and the asserted consistency with AD outputs.","section":"measure-theoretic foundations section bridging coarea formula with conservative Jacobians"}],"minor_comments":[{"comment":"Clarify the precise definition of 'regular' PDL models and the conditions under which the conservative Jacobian is applied, ideally with a dedicated preliminary subsection.","section":"Introduction and preliminaries"},{"comment":"The empirical section on P=2000 Lasso should include explicit mixing-time diagnostics and error bounds for the sampler to support the scaling claims.","section":"empirical evaluation"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the scope of a theoretical ML journal, but the citation list should be checked for completeness regarding prior uses of conservative Jacobians in optimization and geometric measure theory applications to coding."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive report. The single major comment raises a substantive point about invariance under measurable selections of the conservative Jacobian. We address it directly below and commit to a targeted revision that strengthens the measure-theoretic foundations without altering the paper's core claims.","responses":[{"response":"We agree that an explicit invariance argument is not stated as a standalone lemma in the current manuscript, even though the well-posedness claim rests on the uniqueness of the induced Hausdorff measure. In the revised version we will insert a new short lemma (placed immediately after the statement of the extended coarea formula) showing that, for path-differentiable Lipschitz functions, any two measurable selections of the conservative Jacobian differ on a set of Lebesgue measure zero in the parameter space; consequently they generate identical (n-1)-Hausdorff measures on almost every level set. The proof relies on the upper semicontinuity of the conservative Jacobian together with the standard coarea formula applied to the Clarke subdifferential. This addition will also make the consistency with automatic-differentiation outputs fully rigorous, since every AD output lies inside the conservative Jacobian. We view the revision as a clarification rather than a change of substance.","revision_made":"yes","referee_comment":"[measure-theoretic foundations section bridging coarea formula with conservative Jacobians] The extension of the coarea formula to set-valued conservative Jacobians for PDL models asserts a well-posed NML via a unique Hausdorff measure on level sets, but provides no explicit invariance argument or selection theorem showing independence from the choice of measurable selection of the Jacobian. Different selections could yield different measures, directly undermining the well-posedness claim and the asserted consistency with AD outputs."}],"tokens_in":1379,"tokens_out":371,"duration_ms":23706,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The colleague should know two things up front. First, the work supplies a measure-theoretic route to stochastic complexity for regular non-smooth models (Lasso, sparse SVMs) that prior smooth-model coarea approaches could not reach. Second, they supply a practical geometric sampler (PDL-PPMH) that traverses the non-differentiable level sets and report scaling results on a P=2000 Lasso example.\n\nWhat the paper actually does is apply classical geometric measure theory to path-differentiable Lipschitz estimators, replace ordinary Jacobians with conservative ones, and claim the resulting NML is well-posed and matches automatic-differentiation outputs. They also give a stochastic tangent-space proposal plus a convergent non-smooth projection step, then show the sampler mixes on high-dimensional data while the exact NML criterion matches cross-validation predictive performance without data splitting.\n\nThe soft spot sits exactly where the stress-test note flags it. The passage from a measurable selection of the set-valued conservative Jacobian to a unique Hausdorff measure on the level sets is asserted rather than derived in detail in the abstract; different selections could in principle yield different measures, and no invariance argument is indicated. If the full manuscript contains an explicit selection theorem or shows independence, the claim holds; otherwise the well-posedness step remains the least secure. The empirical scaling and robustness checks look reasonable but are secondary to that foundational point.\n\nThis is for people working on information-theoretic model selection or exact alternatives to cross-validation in high-dimensional sparse settings. A reader who needs the sampler or the non-smooth extension will find concrete material; others can skip it.\n\nThe paper deserves a serious referee. The gap it targets is real, the computational contribution is usable, and the measure-theoretic move is worth checking even if the Jacobian selection needs more work.","headline":"This paper extends NML to non-smooth PDL estimators via geometric measure theory and adds a geometric MCMC sampler, but the coarea formula extension to conservative Jacobians needs tighter justification on uniqueness.","tokens_in":2356,"tokens_out":449,"would_cite":false,"duration_ms":23936,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"The stochastic complexity of regular non-smooth models is well-posed and matches automatic differentiation outputs.","keywords":["normalized maximum likelihood","stochastic complexity","non-smooth models","path-differentiable Lipschitz","geometric measure theory","Metropolis-Hastings sampler","Lasso regression","automatic differentiation"],"falsifier":"A direct comparison on a simple Lasso or Sparse SVM example where the NML value obtained from the PDL-PPMH sampler differs from the value produced by automatic differentiation on the same model.","tokens_in":2652,"feed_emoji":"","tokens_out":674,"duration_ms":26607,"temperature":0.7,"pith_summary":"The paper establishes a measure-theoretic definition of the normalized maximum likelihood codelength for non-smooth estimators that dominate modern machine learning. It applies the coarea formula to conservative Jacobians on path-differentiable Lipschitz models, proving that the resulting stochastic complexity integral is rigorously defined and consistent with automatic differentiation. A new geometric MCMC sampler called PDL-PPMH is introduced to compute the quantity exactly by traversing non-differentiable level sets. A reader would care because the method yields a data-efficient model selection criterion that matches cross-validation performance on high-dimensional Lasso without splitting the data.","feed_headline":"Measure theory defines NML for non-smooth estimators","feed_subtitle":"A coarea-based proof and geometric MCMC sampler make stochastic complexity well-posed and consistent with automatic differentiation.","key_machinery":"The coarea formula applied together with conservative Jacobians on regular path-differentiable Lipschitz estimators, which defines the NML integral over non-smooth level sets.","core_discovery":"By bridging the coarea formula with conservative Jacobians for regular path-differentiable Lipschitz estimators, the stochastic complexity for non-smooth models is well-posed and theoretically consistent with the outputs of modern Automatic Differentiation. The Propose-and-Project Metropolis-Hastings sampler enables exact computation by using a stochastic tangent space proposal and a convergent non-smooth projection step, demonstrated on a P=2000 Lasso posterior while showing that the exact NML criterion achieves statistically indistinguishable predictive optima from cross-validation without data splitting.","pith_inferences":["The same measure-theoretic construction may apply to other non-smooth optimization routines used in deep learning.","The consistency result could justify replacing cross-validation with NML in settings where data are scarce.","The mixing-time scaling observed on Lasso may generalize to other PDL models and guide practical sampler tuning."],"forward_implications":["The NML codelength becomes computable for non-smooth estimators such as Lasso and Sparse SVMs.","Exact NML supplies a data-efficient alternative to cross-validation that requires no data splitting.","The sampler scales to high-dimensional problems such as P=2000 while quantifying the exactness-versus-mixing-time trade-off.","The framework supports theoretical analysis of the NML codelength for any regular non-smooth model."],"fun_headline_variants":["NML well-posed for non-smooth models via measure theory","Conservative Jacobians extend NML to PDL estimators","Geometric MCMC samples NML for non-smooth Lasso","Exact NML from PDL-PPMH matches CV without splits"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The estimators must be regular path-differentiable Lipschitz models so that the coarea formula can be applied together with conservative Jacobians.","fun_headline_variants_meta":{"raw":{"variants":["NML well-posed for non-smooth models via measure theory","Conservative Jacobians extend NML to PDL estimators","Geometric MCMC samples NML for non-smooth Lasso","Exact NML from PDL-PPMH matches CV without splits"]},"model":"grok-4.3","cost_usd":0.005949,"raw_usage":{"total_tokens":2862,"prompt_tokens":750,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":59487000,"prompt_tokens_details":{"text_tokens":750,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2044,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":750,"tokens_out":68,"duration_ms":21924,"temperature":1.0,"reasoning_tokens":2044,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T15:09:56.905548+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A direct comparison on a simple Lasso or Sparse SVM example where the NML value obtained from the PDL-PPMH sampler differs from the value produced by automatic differentiation on the same model.","supporting_citations":[],"review_version":1}