{"id":"7e15a8e7-13c2-4477-8707-ce176ec98cc2","arxiv_id":"2501.01127","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"InDeed decomposes images into low-rank, sparse, and noise parts using a network whose modules mirror a hierarchical Bayesian model, and proposes test-time adaptation for out-of-distribution data.","lead":"The paper builds a neural network for splitting an image into low-rank, sparse, and noise components, with each network module tied to a Bayesian model. It claims a generalizability guarantee from a PAC-Bayes bound, but that bound is stated without a valid proof and its form is inconsistent with standard PAC-Bayes inequalities.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 (Eq. 36) is not a valid PAC-Bayes bound: it lacks sample-size and confidence terms, is asserted without proof, and the 'guaranteed generalizability' claim collapses if it fails.","rationale":"The reader's weakest assumption—that Theorem 1 is not a properly stated PAC-Bayes inequality—is exactly the load-bearing concern. The theorem's right-hand side omits the sample size and confidence parameter, which are fundamental to any PAC-Bayes bound. The paper provides no proof, and the surrounding text does not clarify how the standard bounds would reduce to this form. The rest of the theoretical narrative, including the motivation for InDeedAG, depends on this theorem; without it, the method is an unproven heuristic. I agree with the reader's verdict of REJECT because the central claim of 'guaranteed generalizability' is not supported. The empirical work has merit, but it cannot compensate for an invalid theoretical guarantee. The proposed test—re-deriving the bound from a cited PAC-Bayes theorem—would definitively show whether Eq. (36) follows, and I am confident it would expose the missing terms.","tokens_in":32464,"tokens_out":7045,"duration_ms":69388,"concrete_test":"Independently derive Theorem 1 by applying the PAC-Bayes bound of Mbacke et al. (2023) to the variational posterior q(Z|Y)=fθ(Y) with a point-mass prior on θ. The resulting inequality will contain the term sqrt((KL+ln(2√m/δ))/(2m)) on the right-hand side. Show that this term cannot be absorbed into K·d(pHat,p)+C unless K or C depend on m and δ, contradicting the theorem's statement that C is a constant and K depends only on the Lipschitz constant. If the derivation yields a bound with the missing terms, Theorem 1 as stated is false.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of guaranteed generalizability rests on Theorem 1 in Section 3.5.1, which states that for any Kθ-Lipschitz fθ and any δ∈(0,1), with probability at least 1−δ over the sample D, R(fθ) ≤ Rhat(fθ;D) + KLhat(q,p;D) + K·d(pHat(Y),p(Y)) + C. The right-hand side contains no dependence on |D| and no dependence on δ. Standard PAC-Bayes bounds (e.g., McAllester 2003, Germain et al. 2016, Mbacke et al. 2023) always include a term shrinking with √|D| and involving log(1/δ), such as sqrt((KL+ln(2√m/δ))/(2m)). Without such a term, the inequality cannot hold with probability at least 1−δ for all δ, because as δ→0 the required RHS would need to grow, while the stated RHS is fixed. The theorem is asserted without proof, and the cited references do not appear to imply it. Moreover, the paper's actual training loss Eq. (35) includes a supervision term ℓsup, which is not part of the bound, so minimizing Eq. (35) is not equivalent to minimizing the bound. If Theorem 1 is invalid, the claimed connection between the loss and generalization error collapses, and the title's 'guaranteed generalizability' is unsupported. The empirical results, while substantial, do not validate the theory.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InDeed, a framework for interpretable deep image decomposition that combines hierarchical Bayesian modeling (HBM) with a modular neural network whose architecture mirrors the probabilistic graphical model. Variational inference is used to derive two optimization sub-problems, one of which has closed-form solutions; the other is solved by a DNN trained on a negative-ELBO loss. The paper claims a PAC-Bayesian generalization error bound (Theorem 1) showing that minimizing this loss reduces an upper bound on the expected risk, and uses this claimed connection to motivate two test-time adaptation algorithms, InDeedAG and InDeedOAG, for out-of-distribution scenarios. The framework is instantiated on image denoising and unsupervised anomaly detection, with extensive experiments including OOD benchmarks.","tokens_in":32826,"tokens_out":4942,"duration_ms":48403,"significance":"If Theorem 1 were valid, the paper would provide a valuable bridge between variational inference and generalization guarantees for an interpretable, modular architecture, and the proposed test-time adaptation would be well motivated. The experimental portion is substantial and contains useful contributions: closed-form updates for leaf-level variables (Eqs. 26-30), a modular architecture with interpretable intermediate outputs, and an adaptation scheme that improves OOD denoising performance in Table 5. However, the central theoretical claim underpinning both the title and the adaptation motivation is not established; the stated bound is not a correct PAC-Bayes inequality, and the claimed equivalence between the loss and the bound is broken by terms that are actually minimized. The strong empirical results do not repair the collapse of the 'guaranteed generalizability' claim.","major_comments":[{"comment":"Theorem 1 is not a valid PAC-Bayes bound as stated. The right-hand side of Eq. (36) contains no dependence on the sample size |D| and no dependence on the confidence parameter δ, while the theorem asserts the inequality holds with probability at least 1−δ over D for any δ∈(0,1). Every standard PAC-Bayes inequality must include a term that grows with ln(1/δ) and with 1/√|D| (e.g., McAllester's bound), so as δ approaches 0 the right-hand side would need to grow, whereas Eq. (36) is fixed. Moreover, the theorem is asserted without any proof in the manuscript or supplement, and the cited references [22], [23], [69] do not contain this result in this form. Because Theorem 1 is the basis for the title's 'guaranteed generalizability' and for the design of InDeedAG, this is a load-bearing unsupported assertion.","section":"§3.5.1, Theorem 1 and Eq. (36)"},{"comment":"The claimed equivalence between the empirical terms of the bound and the training loss does not hold. Eq. (37) identifies Rhat + KLhat with Ldnn, but the actual loss minimized in Eq. (35) includes the supervision term ℓsup (instantiated in Eq. (44) or (45)), and the full objective in Eq. (43) adds the orthogonality term τℓorth. Even if Theorem 1 were correct, minimizing Eq. (43) is not equivalent to minimizing Rhat + KLhat; the minimized objective contains terms that are absent from the bound and vice versa. Therefore the statement in §3.5.1 that 'minimizing Eq (35) tends to reduce the error bound' is not logically supported by the preceding equations.","section":"§3.5.1, Eq. (37) vs. §3.4.1, Eq. (35)"},{"comment":"The quantities K = c(Kθ) and C are never specified, and d(pHat(Y), p(Y)) is an unknown distribution discrepancy. Consequently Eq. (36) cannot be evaluated, tightened, or controlled from finite data; even a formally correct bound of this form would not provide a practical or verifiable guarantee unless these terms are quantified or bounded. The paper gives no analysis of Kθ or C and no definition of the discrepancy beyond a reference to [23]. As a result, the 'guarantee' is vacuous in its present form, and the relation to the training loss remains symbolic.","section":"§3.5.1, constants K, C and discrepancy d"}],"minor_comments":[{"comment":"The expression L = ABT = Pr0 i=1 ai × bT i is likely a rendering issue; it should be written with an explicit summation sign over i=1 to r0.","section":"§3.1.1, Eq. (16)"},{"comment":"The notation InDeedAGβ (L) and InDeedAGβ (LS) is never defined; the superscript β is unexplained and appears only in the table and surrounding text.","section":"§4.4.1 and Table 5"},{"comment":"The batch size for the main training phase is not reported, while the batch size for adaptation is given as 1 in §4.4; please provide this implementation detail for reproducibility.","section":"§4.1"},{"comment":"The description says Dood is 'sampled from p(Y)', which makes the out-of-distribution setting unclear if p(Y) is also the training distribution; the relationship between the training distribution and the test distribution should be stated more precisely.","section":"§3.5.2"},{"comment":"The scatter plots in Fig. 6 have unlabeled axes; adding axis labels (e.g., ℓrank, ℓsparse, PSNR) would improve readability.","section":"Figure 6"}],"recommendation":"reject","confidential_remarks":"The experimental and architectural content of the paper is solid and could potentially form the basis of a strong empirical or systems-oriented contribution. However, the main advertised contribution—a PAC-Bayesian guarantee of generalizability—rests on an unproven and, as stated, incorrect theorem. This is not a local fixable issue: the theorem's form is incompatible with standard PAC-Bayes bounds, and the connection to the actual training loss is broken by the supervision and orthogonality terms. I would advise the editor that the paper should be rejected in its current form, though the authors might be encouraged to resubmit a revised version that removes or substantially repositions the theoretical claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The framework itself is genuinely interesting: the hierarchical Bayesian low-rank-plus-sparse decomposition, closed-form updates for leaf variables, and a modular network that infers the middle-level posteriors is a coherent recipe, and the low-rank factorization with adaptive rank is a real extension of their earlier BayeSeg work. The test-time adaptation scheme is practical, and the experiments on denoising and anomaly detection are extensive, with competitive results on several OOD benchmarks. That part deserves credit.\n\nThe problem is the title. Theorem 1, which is supposed to give the 'guaranteed generalizability,' is not a legitimate PAC-Bayes bound. The right-hand side has no sample-size dependence and no dependence on the confidence parameter δ; the confidence appears only in the probability statement. Standard PAC-Bayes bounds always contain a term like sqrt((KL + ln(2√m/δ))/(2m)), so as δ→0 the bound must grow. Here the RHS is fixed, so the stated inequality cannot hold for all δ. The theorem is asserted without proof, and the cited references do not imply it. Even the connection to the training loss is circular: Eq. (37) defines Rhat + KLhat as Ldnn, so the 'bound' is mostly a restatement of the training objective plus an unobservable distribution discrepancy and constants. The actual loss also includes a supervision term ℓsup that does not appear in the bound, so minimizing Eq. (35) is not even equivalent to minimizing the bound.\n\nThis is not a minor technical issue. The paper's stated contribution is the theoretical justification for the method and for the test-time adaptation algorithm. If the theorem is wrong, the justification collapses, and the empirical results, while nice, only support a heuristic.\n\nOther soft spots are smaller: no error bars are reported, code is not available (promised only upon acceptance), and some comparisons use retrained baselines where details matter. The anomaly-detection gains in the OOD adaptation section are small, though the visual evidence is plausible.\n\nThe paper is worth a serious referee, but the referee should push hard on the theory. If the authors can either prove a valid PAC-Bayes bound with the correct dependence on sample size and confidence, or explicitly drop the 'guaranteed generalizability' claim and reframe the bound as a heuristic motivation, the paper would be much stronger. I would not cite it in its current form, but I would read a revised version.","headline":"Solid empirical framework, but the 'guaranteed generalizability' claim rests on a PAC-Bayes bound that is not a valid PAC-Bayes bound.","tokens_in":33336,"tokens_out":1713,"would_cite":false,"duration_ms":19287,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Minimizing the variational loss of a Bayesian image-decomposition network is claimed to minimize a PAC-Bayesian upper bound on its generalization error.","keywords":["image decomposition","hierarchical Bayesian modeling","variational inference","PAC-Bayesian generalization bound","test-time adaptation","image denoising","unsupervised anomaly detection","interpretable deep learning"],"falsifier":"Re-derive Theorem 1 with explicit constants and check whether the bound contains a term that shrinks as the training set size $|\\mathcal{D}|$ grows and as $\\delta$ decreases; a concrete falsifier is to compute the bound and the true expected error on a small synthetic denoising task where the expected risk is known in closed form, and show that the inequality fails for some $\\delta$ or dataset size.","tokens_in":32256,"feed_emoji":"🧩","tokens_out":6302,"duration_ms":52498,"temperature":0.7,"pith_summary":"This paper introduces InDeed, a three-step framework for building interpretable deep image-decomposition networks. The network is designed by first writing an image as the sum of low-rank, sparse, and noise components in a hierarchical Bayesian model, then converting variational inference into two optimization sub-problems, and finally learning the hard sub-problem with a modular network whose modules mirror the model. The central claim is that the network's training loss equals the empirical part of a PAC-Bayesian generalization bound, so minimizing the loss is simultaneously minimizing an upper bound on expected error; this is what the paper means by 'guaranteed generalizability.' A test-time adaptation algorithm, InDeedAG/InDeedOAG, is derived from the bound and is reported to improve out-of-distribution denoising and anomaly detection. If the theoretical claim is valid, the framework would give a principled way to design transparent networks with formal OOD guarantees.","feed_headline":"Training loss doubles as a generalization error bound","feed_subtitle":"InDeed splits images into low-rank, sparse, and noise parts, then adapts to new domains without labels.","key_machinery":"The load-bearing object is the variational training loss $L_{\\mathrm{dnn}} = \\mathbb{E}_{p(Y)}[-\\mathrm{ELBO}(q_{f_\\theta(Y)}(Z);Y)]$, equivalently the sum of fidelity, supervision, low-rank, sparsity, and prior KL terms in Eq. (35). The argument that this loss controls generalization runs through Theorem 1, a PAC-Bayesian bound patterned on references [22], [23], [69], whose right-hand side contains the empirical loss plus the empirical KL between variational posterior and prior, so that $\\hat{L}_{\\mathrm{dnn}}$ appears as the empirical part of the bound. The network architecture is also machinery: $f_\\theta$ has a low-rank module $f_{\\theta_L}$ with two parallel sub-networks predicting mean/variance of factors $A$ and $B$, and a sparsity module $f_{\\theta_S}$ predicting mean/variance of $S$ from the residual $Y - \\hat{A}\\hat{B}^T$; the leaf-level variables $\\gamma$, $\\Omega$, $\\Lambda$ are updated in closed form by conjugate-prior formulas (Eqs. (26), (28), (30)).","core_discovery":"InDeed models an image observation as $Y = L + S + N$, with $L = AB^T$ a low-rank product, $S$ sparse, and $N$ pixel-wise Gaussian noise, all governed by conjugate priors (Gamma priors on precision-like variables to control rank and sparsity). Variational inference over the posterior $q(Z) = q(A)q(B)q(S|A,B)q(\\gamma)q(\\Omega)q(\\Lambda)$ splits the negative ELBO into seven additive terms, of which the leaf-level variables ($\\gamma$, $\\Omega$, $\\Lambda$) have closed-form updates while the middle-level variables ($A$, $B$, $S$) are predicted by a modular network $f_\\theta$. The paper's main theoretical assertion is Theorem 1, which states that the expected generalization error $R(f_\\theta)$ is bounded, with probability at least $1-\\delta$, by the sum of the empirical error, the empirical KL term, a distribution-discrepancy term $K\\cdot d(\\cdot,\\cdot)$, and a constant $C$; because the first two terms are exactly the network's training loss $\\hat{L}_{\\mathrm{dnn}}$, the authors conclude that minimizing $\\hat{L}_{\\mathrm{dnn}}$ controls generalization error. From this bound they derive an unsupervised test-time adaptation method that fine-tunes selected modules on out-of-distribution data, and they report gains over strong baselines in denoising (SIDD, PolyU) and anomaly detection (noisy MVTecAD, Severstal, MOOD).","pith_inferences":["Inference: A standard PAC-Bayes bound must contain a term that decays with the training sample size $|\\mathcal{D}|$ and scales with $\\log(1/\\delta)$; Eq. (36) as printed shows neither, so whether the claimed guarantee is a valid PAC-Bayes result is a question that determines the strength of the paper's central assertion.","Inference: The distribution-discrepancy term $K\\cdot d(\\hat{p}_{\\mathcal{D}}(Y),p(Y))$ suggests that test-time adaptation implicitly minimizes an empirical distribution shift, and a direct measurement of $d(\\cdot,\\cdot)$ before and after adaptation would make that mechanism explicit.","Inference: The modular separation of low-rank and sparsity modules may support transfer learning, for instance reusing $f_{\\theta_L}$ across tasks with different noise types; the paper mentions this as future work, so it is an untested consequence."],"forward_implications":["If Theorem 1 is valid, then any architecture that minimizes $\\hat{L}_{\\mathrm{dnn}}$ inherits a PAC-Bayesian guarantee, making the claimed generalizability a formal property rather than an empirical hope.","The test-time adaptation algorithm InDeedAG follows directly from the bound: fine-tuning only the sparsity module on unlabeled OOD data reduces the empirical loss and therefore the bound, which matches the reported improvements on SIDD and PolyU.","The modularized architecture means that adaptation can be targeted to the module most responsible for the distribution shift (in these experiments, the sparsity module $f_{\\theta_S}$), offering a cheap, selective fine-tuning strategy.","The framework's interpretability is a by-product of the same construction: each loss term and each intermediate output corresponds to a named variable in the probabilistic model, so the decomposition $L$, $S$, $N$ is directly inspectable."],"supporting_citations":[{"why":"Supplies the PAC-Bayesian margin-bound framework that Theorem 1 is based on.","marker":"[22]"},{"why":"Defines the distribution discrepancy $d(\\cdot,\\cdot)$ and the PAC-Bayes bound for variational autoencoders, from which the theorem's third term is taken.","marker":"[23]"},{"why":"Connects PAC-Bayesian theory to Bayesian inference, supporting the identification of the training loss with the bound's empirical terms.","marker":"[69]"},{"why":"Provides the reparameterization trick used to sample from variational posteriors in the fidelity and supervision loss terms.","marker":"[67]"},{"why":"Supplies the mean-field variational inference framework that splits the posterior and yields the two sub-problems.","marker":"[61]"},{"why":"Establishes the robust PCA model of low-rank plus sparse decomposition that the generative model extends.","marker":"[6]"}],"fun_headline_variants":["InDeed: training loss doubles as a generalization bound","Guaranteed generalization via loss bound in InDeed","Interpretable deep decomposition with provable generalization","Loss bound predicts generalization in InDeed","Test-time adaptation from a theoretical loss bound"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Theorem 1 must be a genuine PAC-Bayesian inequality with constants that depend on the sample size and the confidence parameter; if the derivation is invalid, the paper's guarantee of generalizability reduces to an unproven heuristic.","fun_headline_variants_meta":{"raw":{"variants":["InDeed: training loss doubles as a generalization bound","Guaranteed generalization via loss bound in InDeed","Interpretable deep decomposition with provable generalization","Loss bound predicts generalization in InDeed","Test-time adaptation from a theoretical loss bound"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00062,"raw_usage":{"total_tokens":2931,"prompt_tokens":1059,"completion_tokens":1872,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":1800}},"tokens_in":675,"tokens_out":1872,"duration_ms":13078,"temperature":1.0,"reasoning_tokens":1800,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:34:54.792592+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-derive Theorem 1 with explicit constants and check whether the bound contains a term that shrinks as the training set size $|\\mathcal{D}|$ grows and as $\\delta$ decreases; a concrete falsifier is to compute the bound and the true expected error on a small synthetic denoising task where the expected risk is known in closed form, and show that the inequality fails for some $\\delta$ or dataset size.","supporting_citations":[{"cited_title":"Simplified pac-bayesian margin bounds,","cited_arxiv_id":null,"evidence_quote":"Supplies the PAC-Bayesian margin-bound framework that Theorem 1 is based on."},{"cited_title":"Statistical guarantees for variational autoencoders using pac-bayesian theory,","cited_arxiv_id":null,"evidence_quote":"Defines the distribution discrepancy $d(\\cdot,\\cdot)$ and the PAC-Bayes bound for variational autoencoders, from which the theorem's third term is taken."},{"cited_title":"Pac- bayesian theory meets bayesian inference,","cited_arxiv_id":null,"evidence_quote":"Connects PAC-Bayesian theory to Bayesian inference, supporting the identification of the training loss with the bound's empirical terms."},{"cited_title":"Variational infer- ence: A review for statisticians,","cited_arxiv_id":null,"evidence_quote":"Supplies the mean-field variational inference framework that splits the posterior and yields the two sub-problems."}],"review_version":1}