{"id":"79350c40-05d1-4d26-8994-d70d7e654638","arxiv_id":"2512.08444","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Learned iterative reconstruction networks can be uniformly described as operator learning: the unrolled architecture fixes how to compute while the loss and data fix what to compute; for nonlinear inverse problems the update direction matters most.","lead":"This chapter surveys learned iterative reconstruction networks—neural networks built by unrolling classical optimization algorithms—and casts them as operator learning. It separates architecture choice from the training objective and shows numerically that the choice of update direction matters most for nonlinear inverse problems.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Nonlinear update-direction claim rests on an imported, unreproduced diffusion-approximation comparison","rationale":"The central conceptual claim — that learned iterative networks can be viewed as unrolled operators and that many architectures differ mainly in the parametrization of the neural updating operator — is argued with explicit equations and is internally consistent; I do not see a load-bearing mathematical flaw there. The weakest point is the numerical evidence used to make the stronger empirical claim about nonlinear problems. The reader's weakest_assumption correctly identifies that the QPAT comparison is imported and may not be representative. I push one step further: the manuscript does not even include the promised Figure 3 or any quantitative summary of that comparison, and the diffusion approximation rather than radiative transfer is used. Since Section 11 explicitly contrasts 'minor difference' for linear problems with 'major effect' for nonlinear problems, this nonlinear result carries substantial weight. A single reproduction under a second forward model with proper seed variance would settle whether the ranking is a property of update directions or an artifact of the approximate model. Until then, the survey's framework can be accepted, but the nonlinear performance claim should remain conditional.","tokens_in":42024,"tokens_out":5236,"duration_ms":61336,"concrete_test":"Reproduce the Section 10.2 QPAT comparison under two forward models — the diffusion approximation (Eq. 57) and a radiative-transfer-based solver or a substantially different discretization — using the same ResNet updating operator, iteration count, parameter count, training protocol, and at least 5 random seeds. If the learned Gauss-Newton > GD/SR1 ranking does not persist across forward models and seeds, or if the differences fall within seed variance, the Section 11 conclusion should be weakened to a conditional statement.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's practical conclusion in Section 11 — that in nonlinear inverse problems the choice of update direction has a major effect on performance — is supported only by Section 10.2, which imports the QPAT comparison from the author-associated preprint [108]. The section states the results are in Figure 3, but no figure, table, error bars, or code are included in this manuscript. The forward model is the diffusion approximation (Eq. 57), an acknowledged simplification of radiative transfer; the ranking of GD, Gauss-Newton, and quasi-Newton SR1 update directions may depend on the curvature and conditioning of this approximate model and on training hyperparameters. Without an independent reproduction, or at least reported variance and model-mismatch sensitivity, the claim that update direction is a major factor for nonlinear problems is not established by the evidence presented here.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified operator-learning perspective for learned iterative networks in inverse problems. It defines a learned reconstruction operator and separates the 'how to compute' (architecture) from the 'what to compute' (learning problem), then surveys learned gradient networks, learned proximal networks, variational networks, learned primal-dual networks, and extensions to nonlinear problems and Newton-type methods. The main structural claim is that popular unrolled architectures—learned least squares, proximal gradient, variational networks, and learned primal-dual—are special cases of a common abstract update (Eqs. (17), (20), (42)). A short numerical study is presented to compare update formulations for linear CT and to report a comparison for nonlinear QPAT.","tokens_in":42241,"tokens_out":10114,"duration_ms":98752,"significance":"If the survey's framework is accepted, it provides a genuinely useful conceptual simplification: it identifies the statistical estimator as determined by the loss and training data, while the architecture determines approximation capacity and generalization behavior. The paper's strength is that the containment claims are given explicitly: Eq. (18), (19), (35), and Remark 7.1 show how specific architectures arise from the general update. The functional-analytic gradient and Hessian calculations in Appendix A are standard and correct. The survey also covers topics often omitted from such reviews, including PnP/DEQ connections, greedy training, and Newton-type unrolling. The numerical evidence is less conclusive and needs to be strengthened or qualified.","major_comments":[{"comment":"The conclusion that in nonlinear inverse problems 'the choice of update direction ... has a major effect on the performance' is supported only by the QPAT discussion in §10.2. The text states that 'the results are presented in Figure 3', but no Figure 3 appears in the manuscript; no numerical table, error bars, or experimental details are included. The forward model is also the diffusion approximation (Eq. (57)), and the entire study is imported from the preprint [108]. As written, this is not sufficient evidence for the strong conclusion in §11. The manuscript should either include the actual comparison (with variance or error bars) or clearly present it as a reported result from [108] and qualify the conclusion accordingly.","section":"§10.2 and §11"},{"comment":"The formulas labeled BFGS are not the BFGS update formulas as defined. With H_k denoting a Hessian approximation and B_k denoting an inverse-Hessian approximation, the displayed B_k update is the DFP inverse-Hessian update, not the BFGS inverse-Hessian update; the H_k update is the BFGS inverse-Hessian update if H_k were the inverse Hessian, which contradicts the definition in Eq. (49). This is a technical error in a section that is meant to give readers correct quasi-Newton formulas and their Hilbert-space versions. Please correct the formulas and verify them against [143].","section":"§8.2.3, Eqs. (51)–(52)"},{"comment":"The linear numerical comparison reports average PSNR over 50 test samples without error bars or repeated runs. The differences between the learned gradient networks are 0.05–0.23 dB, which is likely within run-to-run variation for the same architecture. The subsequent statements that 'performance increases with decreasing structure' and that LPD 'clearly' improves visually should be supported by error bars, statistical significance, or at least qualified as preliminary. If this is intended only as an illustrative study, the wording in the text should make that explicit.","section":"§10.1, Table 1"}],"minor_comments":[{"comment":"Eq. (19) writes Λ_θ(f,∇Q_g(f)) := Γ_θ(f−∇Q_g(f)), but the left-hand side is a two-argument map and the right-hand side is a one-argument map. This is presumably a shorthand in which the first argument is ignored. Please state explicitly that the proximal update is obtained from the two-argument Λ_θ by Λ_θ(a,b)=Γ_θ(a−b), so that the containment of Eq. (29) in Eq. (17) is exact.","section":"§6, Eq. (19)"},{"comment":"There is an indexing mismatch in the special case discussed after Eq. (48): the general form uses f_k := Λ^k_{θ_k}(f_0, Δf_0, ..., f_{k-1}, Δf_{k-1}), but the no-memory special case writes f_{k+1} := Λ_{θ_k}(f_k, ...). Please align the indices for clarity.","section":"§8.2.2, Eq. (48)"},{"comment":"The proximal operator prox_S is stated as 'prox_S : X → R', but it is a map X → X. Please correct the type.","section":"§6.1.2, Eq. (28)"},{"comment":"The statement 'f_k := 0 if k < 0' is not a standard initialization condition. It likely should be 'f_k := 0 for k < 0' or should define the initial boundary values more carefully. Please clarify.","section":"§6.1.1, Eq. (23)"},{"comment":"Equation (62) has a typographical error: the subscript on the second inner product is 'E' but should be X or Y. Also, the reference [118] in the bibliography contains the typo 'Lunz ansd Okan Oktem'; please fix.","section":"Appendix A, Eq. (62)"}],"recommendation":"major_revision","confidential_remarks":"The survey is well organized and the core conceptual claim is sound, but the nonlinear numerical evidence is currently missing from the manuscript (no Figure 3) despite being used for a key conclusion. The quasi-Newton formula error in §8.2.3 should be corrected before publication. The heavy reliance on the authors' own recent preprint [108] for the central nonlinear claim is worth flagging; if this is a survey, the evidence should be either independently reproducible or reported with appropriate caveats."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a survey, and a useful one. The core message—separate the reconstruction operator (how to compute) from the learning problem (what to compute)—is stated clearly and backed with explicit equations. The equivalence of variational, proximal, least-squares, and even learned primal-dual as special cases of a generic unrolled operator is worked out concretely (Eqs. 17–22, 35, and Remark 7.1). That is real value: someone new to unrolling could use this to see the common structure behind a lot of literature. The appendix gradient/Hessian calculations are correct and standard.\n\nThe soft spots are mostly in the numerical section. Section 10.1 compares four architectures under roughly matched parameter counts and reports PSNR differences of 0.05–0.2 dB with no error bars and no code or data. The authors are honest that protocols were not optimized per method, so the ranking should not be over-read. My bigger concern is Section 10.2. The paper's headline practical conclusion—that in nonlinear inverse problems the update direction has a major effect—rests entirely on a QPAT comparison imported from the authors' own preprint [108]. The results are described but the figure is not included in this manuscript, no error bars or sensitivity analysis are given, and the underlying diffusion approximation is a known simplification. In other words, the central empirical claim for nonlinear problems is asserted but not evidenced in this paper. That does not mean the claim is false; [108] may well support it. But as this manuscript stands, a reader cannot check it.\n\nThe structural survey part is sound and will be useful to a specific audience: researchers who want a compact, equation-level map of unrolled methods and the conceptual point about loss versus architecture. I would send it to peer review, because the survey has real utility, but I would ask the authors to either make the nonlinear comparison reproducible (include the figure with variance, state the evaluation protocol, or point to a public code) or soften the conclusion accordingly. If they cannot do that, the practical claim should be labeled as preliminary.","headline":"A clean operator-learning survey of unrolled networks whose practical claim about nonlinear update directions rests on an imported comparison that does not appear in this manuscript.","tokens_in":42661,"tokens_out":2219,"would_cite":true,"duration_ms":23760,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65J22","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that learned iterative reconstruction networks are all special cases of one abstract operator, so the loss, not the architecture, determines what they compute.","keywords":["learned iterative networks","operator learning","algorithm unrolling","inverse problems","learned reconstruction operator","variational networks","learned primal-dual","learned gradient networks"],"falsifier":"Re-run the linear CT comparison using an out-of-distribution test set and per-algorithm optimized hyperparameters; if the primal-dual advantage disappears or reverses, the claim that architecture is secondary for linear problems weakens. Re-run the QPAT comparison with the radiative transfer equation instead of the diffusion approximation; if the Gauss-Newton advantage over gradient and quasi-Newton updates vanishes, the nonlinear result may be an artifact of model mismatch.","tokens_in":41942,"feed_emoji":"🔁","tokens_out":6518,"duration_ms":56657,"temperature":0.7,"pith_summary":"This chapter is a survey, but it makes a concrete scientific claim: the many learned iterative reconstruction networks in the literature are not genuinely different algorithms. If you write down the learned reconstruction operator—the recursive map obtained by truncating an iterative scheme and replacing handcrafted updates with neural networks—then learned gradient, learned proximal, variational, and learned primal-dual networks are all special cases that differ only in how the neural updating operator is parametrized. The authors split this into two independent questions: the reconstruction operator says how to compute, and the learning problem (loss plus training data) says what to compute. The practical payoff of the framework is twofold: it explains why unrolling more iterations does not mean converging to the true minimizer, and its numerical experiments indicate that for nonlinear inverse problems the choice of handcrafted update direction matters more than the network architecture.","feed_headline":"Loss, not architecture, sets what unrolled networks compute","feed_subtitle":"A unified operator view: variational, proximal, and primal-dual networks differ only in how they parametrize one update.","key_machinery":"The machinery is the abstract unrolled recursion Rθ(g)=f_N with f_k=Λ^k_{θ_k}(g,f_0,…,f_{k−1}), in which each handcrafted update of an iterative scheme is replaced by a neural operator Λ^k_{θ_k}. The paper treats this operator formulation as independent of both discretization and learning problem. The workhorse inside the recursion is the neural updating operator, typically an image-to-image CNN; the update direction ∇Q_g(f)=A*(Af−g) (or its nonlinear analogue) is what injects the forward model. Different architectures then correspond to different placements of Λ around the update direction: variational networks use f−∇Q+Γθ(f), learned proximal networks use Γθ(f−∇Q), and learned primal-dual","core_discovery":"The core discovery is that a single abstraction—the learned reconstruction operator Rθ:Y→X defined by f_k = Λ^k_{θ_k}(g, f_0,…,f_{k−1}) for k=1,…,N—captures essentially every learned iterative network. In this abstraction the update scheme and the learning problem are decoupled. The authors show that variational networks, learned proximal networks, learned least-squares networks, and learned primal-dual networks are all obtained from this recursion by selecting a particular form for the neural updating operator: the primal-dual architecture adds a second neural operator in data space, but it reduces to a learned least-squares network when the dual update is fixed to Γ(h,h′,h″)=h′−h″. Because","pith_inferences":["If the separation of 'how to compute' from 'what to compute' holds, comparisons between reconstruction networks should control for the learning problem before crediting architecture; otherwise apparent gains may be estimator changes, not design wins.","The nonlinear result suggests a cheap screening strategy: choose the handcrafted update direction (gradient vs Gauss-Newton vs quasi-Newton) using short greedy-trained runs before committing to expensive end-to-end training.","The linear result that less handcrafted structure performs slightly better in-distribution implies the benefit of unrolling for linear problems is mostly computational and generalization-oriented; a proper out-of-distribution test would reveal whether the architectural structure actually helps.","The framework predicts that swapping the loss function on a fixed unrolled network should move the learned estimator toward the corresponding Bayes estimator; this could be tested by training the same architecture with L2, L1, and adversarial losses on the same data."],"forward_implications":["Unrolling more iterations does not by itself bring a network closer to the minimizer of the underlying variational problem; it increases capacity.","The loss function and training data, not the unrolled architecture, determine the statistical estimator: L2 loss approximates conditional expectation, L1 loss conditional median.","Learned primal-dual networks are an extension of learned gradient networks; they reduce to a learned least-squares network when the dual update is fixed.","For linear inverse problems, the choice among gradient-based unrolling formulations changes PSNR by less than 0.1 dB under comparable settings, whereas for nonlinear problems the update direction has a major effect."],"fun_headline_variants":["One operator unifies all unrolled networks","Unrolled nets: same core, different update","Learned iterative nets share one abstraction","What separates unrolled networks? Just the update"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The numerical conclusions assume that matching parameter counts and training protocols isolates the effect of the update direction, and that the diffusion-approximation QPAT test stands in for nonlinear inverse problems in general.","fun_headline_variants_meta":{"raw":{"variants":["One operator unifies all unrolled networks","Unrolled nets: same core, different update","Learned iterative nets share one abstraction","What separates unrolled networks? Just the update"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00049,"raw_usage":{"total_tokens":2204,"prompt_tokens":655,"completion_tokens":1549,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":399,"completion_tokens_details":{"reasoning_tokens":1502}},"tokens_in":399,"tokens_out":1549,"duration_ms":11533,"temperature":1.0,"reasoning_tokens":1502,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T17:39:17.075392+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the linear CT comparison using an out-of-distribution test set and per-algorithm optimized hyperparameters; if the primal-dual advantage disappears or reverses, the claim that architecture is secondary for linear problems weakens. Re-run the QPAT comparison with the radiative transfer equation instead of the diffusion approximation; if the Gauss-Newton advantage over gradient and quasi-Newton updates vanishes, the nonlinear result may be an artifact of model mismatch.","supporting_citations":[],"review_version":1}