{"id":"9bb60755-0dd9-4302-bc15-96204f0d6ee3","arxiv_id":"2509.08359","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A gradient-perturbation method that combines prediction-loss and decision-loss gradients, with a decaying weight, to stabilize decision-focused learning.","lead":"This paper introduces a training trick for decision-focused learning: instead of using only the decision loss gradient, it blends it with the prediction loss gradient, with the prediction influence decaying over time. The result is a plug-in updater that claims to give stable training and lower regret on knapsack, budget allocation, and portfolio problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.4's proof does not establish convergence to a Pareto stationary point: it only shows lim inf of a stationarity measure is zero, and silently assumes boundedness of m_k; the claimed O(1/√T) rate is also contradicted by the proof's own bound.","rationale":"The reader's weakest assumption concerned the gap between exact decision loss gradients required by the theory and the surrogate gradients used in the discrete experiments. That is a legitimate limitation, and the paper acknowledges it in Section 5. However, the more fundamental issue is that Theorem 3.4's proof—the paper's headline theoretical contribution—does not actually establish the stated conclusion even when exact gradients are available. The proof's final inference from a liminf of a product to existence of a limit point that is Pareto stationary is a non-sequitur without compactness. The proof also silently assumes m_k is bounded. The rate claim is not merely imprecise; it is backwards. These are correctness risks in the core argument, and they are not acknowledged in the paper. My read is that the verdict remains CONDITIONAL: the empirical results may be valuable, and the proof may be fixable with additional assumptions (e.g., bounded iterates or bounded stationary measure), but as written the central theoretical guarantee is unverified. Therefore I do not change the reader's verdict, but I identify a different load-bearing concern than the reader's weakest_assumption.","tokens_in":12938,"tokens_out":12012,"duration_ms":122790,"concrete_test":"Re-derive the proof of Theorem 3.4 with the explicit goal of proving convergence of the iterates (not merely lim inf). Determine whether the assumptions given are sufficient to establish that a subsequence of {x_k} converges to a finite Pareto stationary point. If the proof requires an unstated boundedness assumption (e.g., m_k ≤ A or sup_k ||x_k||<∞), then the theorem is not proven as stated; this directly settles the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim (Theorem 3.4) is not supported by the proof. The proof derives lim inf_{k→∞} (m_k ψ_k)=0 from the step-size conditions, where m_k ψ_k is a nonnegative stationarity measure. It then states that along some subsequence either ||∇L_pred||→0, ||∇L_dec||→0, or cos(φ_k/2)→0, 'each implying Pareto stationarity.' This inference requires that the subsequence of iterates converge to a finite point. No such convergence is shown; Lipschitz continuity of gradients does not imply boundedness of iterates. The proof also assumes m_k = sqrt(||∇L_pred|| ||∇L_dec||) ≤ A, an unstated assumption not in the theorem statement. Standard nonconvex analysis needs a compactness/decay argument to extract a limit point. Additionally, the claimed rate O(1/√T) is mathematically reversed: the proof's bound is min_{0≤k<T}(m_kψ_k) ≤ O(T^{-(1-α)}) with α∈(1/2,1); since 1−α<1/2, T^{-(1−α)} is asymptotically larger than T^{-1/2}, so the statement '< O(1/√T)' is false. Thus the main theoretical guarantee—convergence to a Pareto stationary point—is unproven as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a gradient-perturbation method for decision-focused learning (DFL) in which the prediction-loss gradient is used to guide the decision-loss gradient. The update is g = m (α u_pred + u_dec)/||α u_pred + u_dec||, with α a sigmoid-like decay in the epoch number and m the geometric mean of the two gradient norms. The authors claim that the update never conflicts with either gradient when κ=0, that the κ=0 variant converges to a Pareto stationary point, and that the method improves training stability and regret across knapsack, budget allocation, and portfolio optimization problems. The theory is proved only for κ=0, while the experiments also use the decaying κ=1 variant. The manuscript includes a code link and reports normalized test regret with standard errors for five settings.","tokens_in":13287,"tokens_out":7476,"duration_ms":94377,"significance":"If the convergence claim were correct, the method would be a simple, plug-and-play addition to existing DFL pipelines: it requires no surrogate loss, no additional training, and only one extra gradient evaluation. The empirical study covers standard benchmarks and the code is made available. However, the central theoretical contribution is not established in the submitted form: the proof of Theorem 3.4 contains a false rate comparison and an unjustified convergence step, and the theorem covers only the κ=0 case while the method is also used with κ=1. The empirical gains over strong baselines are often small and not statistically verified. The idea is still worth pursuing, but the current manuscript overstates both its theoretical and empirical support.","major_comments":[{"comment":"The proof does not establish convergence to a Pareto stationary point. It shows only lim inf_{k→∞}(m_k ψ_k)=0, then concludes that along a subsequence one of three conditions holds and that this implies Pareto stationarity. This inference requires a convergent subsequence of iterates; Lipschitz continuity of gradients does not imply boundedness of the iterates, so no limit point is guaranteed. In addition, the proof introduces the assumption m_k ≤ A without stating it in the theorem. The rate claim is also reversed: for α∈(1/2,1), T^{-(1−α)} is asymptotically larger, not smaller, than T^{-1/2}, so the statement '< O(1/√T)' is false. The theorem as written is therefore unsupported.","section":"Appendix A, proof of Theorem 3.4"},{"comment":"Theorem 3.4 covers only κ=0, i.e., a fixed bisector direction. The decayed variant κ=1, which is used in the experiments and motivated by the method description, has no convergence guarantee. As t grows, α tends to 0 and the update direction approaches u_dec; the behavior of this time-varying direction is not analyzed. The conclusion's statement that 'our method' converges to a Pareto stationary point is therefore overbroad. Please state precisely which variant is covered by which guarantee.","section":"Sections 3.3–3.4 and 5"},{"comment":"The empirical claim 'our approach is the only method that consistently outperforms across all tasks' is not supported. In the unweighted knapsack setting, the best regret is 0.063±0.040 (convex combination, PCGrad, MGDA), while the proposed method achieves 0.066±0.047 (κ=0) and 0.068±0.047 (κ=1), so the proposed method is not the best in that task. Moreover, the standard errors overlap for many entries, and no statistical tests are reported. Separately, the theoretical assumptions require differentiable exact decision-loss gradients, but the knapsack and budget-allocation experiments use binary decisions and surrogate gradients; this is acknowledged in the limitations, but it means the experiments do not directly validate the convergence theorem.","section":"Table 1 and Section 4.2"}],"minor_comments":[{"comment":"Typo: 'gradeints' should be 'gradients'.","section":"Definition 3.1"},{"comment":"The symbol α is used both for the sigmoid-like decay parameter in Eq. (4) and for the step-size exponent in the proof of Theorem 3.4. This is confusing; please use different symbols.","section":"Equation (4) and Appendix A"},{"comment":"The proof says to stop if u_pred + u_dec = 0, but the update is undefined if either individual gradient is zero. Even if a zero gradient is Pareto stationary, the proof should state a convention for handling this case.","section":"Appendix A, proof of Theorem 3.4"},{"comment":"The table header contains 'PCGR A D' (should be 'PCGrad'). The β values for the convex-combination rows are not aligned with the row labels in the rendered table, making it hard to read.","section":"Table 1"},{"comment":"'An edge case worth exploring further.' is a sentence fragment; consider rephrasing as a complete sentence.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful empirical idea and a clear presentation, but the main theorem is not proved as stated. I would ask for a corrected or weakened Theorem 3.4, a clear separation of the κ=0 and κ=1 guarantees, and a more careful reading of the empirical results. The rate claim in particular cannot be fixed by a minor edit; the authors need to either prove a valid rate under stronger assumptions or remove the rate claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper for its method, not for its theorem. The update rule in Eqs. (4)–(6)—unit-gradient bisection, geometric-mean magnitude, sigmoid-shaped decay—is genuinely new in the DFL context. It is simple, plug-and-play, adds no training overhead, and the paper correctly positions it against PCGrad, MGDA, and DCGD. The geometric analysis of why naive convex combinations fail (near-zero decision gradients with conflicting directions) is useful, and the code is linked. So there is real substance here.\n\nThe soft spots are in the theory, and they are load-bearing. Theorem 3.4 says κ=0 converges to a Pareto stationary point and gives O(1/√T). The proof only establishes lim inf (m_k ψ_k)=0. Passing from that to a Pareto stationarity claim needs bounded iterates; the proof never shows the sequence is bounded. It also silently assumes m_k ≤ A, which is not in the theorem statement. The rate claim is worse: the proof's own bound is O(T^{-(1-α)}) with α∈(1/2,1), and since 1−α < 1/2, T^{-(1−α)} is asymptotically larger than T^{-1/2}. Writing “< O(1/√T)” is backwards. These are fixable if you add a compactness or boundedness assumption, but as stated the main guarantee does not hold.\n\nOn the empirical side, the method wins in most settings but the wins are frequently within standard error, and it loses on unweighted knapsack. The phrase “consistently outperforms” is too strong. Also, the theory assumes exact decision-loss gradients, while the knapsack and budget experiments use surrogate gradients from discrete solvers; the authors acknowledge this in the conclusion, so it is not hidden, but it does mean the theory does not cover the empirical setting. The recommended κ=1 variant has no convergence guarantee at all, which is worth saying plainly.\n\nWho is this for? People working on predict-then-optimize and decision-focused learning. The method is worth discussing and probably worth citing as related work, even though the theoretical justification needs repair. I would send it to peer review—a serious referee can ask for the theorem to be fixed, the rate restated, and the empirical claims toned down. The idea is simple enough that this could become a useful part of the DFL toolbox once the analysis is cleaned up.","headline":"The angle-bisecting update is a neat idea, but the main convergence theorem as written is not proven—the proof only shows lim inf of a stationarity measure is zero, assumes boundedness without saying so, and the claimed O(1/√T) rate is contradicted by the proof's own bound.","tokens_in":13789,"tokens_out":3938,"would_cite":true,"duration_ms":48129,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Blending prediction and decision gradients makes decision-focused learning converge to a Pareto-stationary point with lower regret.","keywords":["decision-focused learning","predict-then-optimize","gradient perturbation","Pareto stationarity","multi-task gradient conflict","end-to-end learning","stochastic optimization","regret"],"falsifier":"For a smooth problem with exact decision gradients (portfolio optimization is one), run the kappa=0 update and record at every epoch the quantity the proof calls m_k psi_k: the combined progress toward both losses along the update direction. Theorem 3.4 predicts the minimum of this quantity over the first T epochs shrinks as O(T^-(1-alpha)) and tends to zero. If it stops decreasing while both individual gradient norms are clearly nonzero and their angle is below pi, the Pareto-stationary guarantee is falsified. A separate quick check: the paper's Proposition 3.2 says the merged update never ha","tokens_in":12832,"feed_emoji":"🎯","tokens_out":10511,"duration_ms":104932,"temperature":0.7,"pith_summary":"Decision-focused learning (DFL) trains a predictor by pushing gradients through a downstream optimizer, but its loss surface is flat in parts and sharp elsewhere, so plain DFL is unstable. This paper proposes to fix that by perturbing the decision-loss gradient with the prediction-loss gradient: early in training the update direction bisects the angle between the two gradients, then gradually shifts toward the decision loss as training proceeds. The merge uses a geometric-mean scale so neither gradient dominates, and a sigmoid-like decay parameter controls the handoff. The authors prove that with the decay switched off (kappa=0), the update converges to a point where neither loss can be improved without hurting the other, at an O(1/sqrt T) rate under smoothness assumptions. On knapsack, budget-allocation, and portfolio problems, the method reports the lowest normalized regret in most settings with consistently small standard errors.","feed_headline":"Prediction gradients steer decision-focused learning to lower regret","feed_subtitle":"A plug-in gradient blend beats PFL and DFL on normalized regret, with more stable runs.","key_machinery":"The central object is the merged gradient g = m * (alpha * u_pred + u_dec) / ||alpha * u_pred + u_dec||, where u_pred and u_dec are unit directions of the prediction and decision loss gradients, m is the geometric mean of their norms, and alpha is a sigmoid-like decaying parameter with an inflection point and steepness kappa. The bisecting-plus-decay construction guarantees, by Proposition 3.2, that the update never conflicts with the decision-loss gradient; with kappa=0 it never conflicts with either gradient. The geometric-mean scaling prevents the usually much larger prediction gradient from dominating the sum. The proof of Theorem 3.4 uses the aggregate loss L_pred + L_dec and shows that","core_discovery":"On its own terms, the paper claims that the instability of vanilla DFL can be removed without surrogate losses and without extra training: simply replace the raw decision-loss gradient with a geometric-mean-scaled vector that bisects the angle between the prediction-loss and decision-loss gradients, then optionally let a decaying weight pull the direction toward the decision loss over epochs. With the decay switched off (kappa=0), the update never conflicts with either gradient, and Theorem 3.4 states that under Lipschitz-smooth losses and standard step-size conditions the iterates reach a Pareto-stationary point of L_pred and L_dec at rate O(1/sqrt T). The empirical claim is comparative: ac","pith_inferences":["An untested consequence: because the method works on discrete problems despite the exact-gradient assumption, it may act as a regularizer on surrogate-gradient DFL more broadly; a controlled study varying solver approximation quality would separate the method's geometric effect from the solver's noise.","The geometric-mean scale means the update magnitude shrinks toward zero when either gradient vanishes; near a point where one loss is flat, the method may slow down while a convex combination would still move. This late-training behavior is not analyzed in the paper.","The same angle-bisecting rule could be applied to other multi-objective training settings where one gradient is more reliable early and another is the true target, such as physics-informed or safety-constrained training; the paper only demonstrates predict-then-optimize problems."],"forward_implications":["Any existing differentiable DFL solver can be wrapped with this update rule; no surrogate loss model or extra training loop is needed.","The kappa=0 variant carries a convergence guarantee to a Pareto-stationary point at O(1/sqrt T), matching the guarantees of gradient-surgery multi-task methods.","Because the update never conflicts with the decision-loss gradient, even when prediction and decision gradients disagree strongly and differ in scale by 10 to 1000 times, the decision loss can still improve.","On knapsack, budget allocation, and portfolio problems the method reports lower normalized regret with lower standard error than PFL, DFL, convex combinations, and gradient-perturbation baselines.","When either PFL or DFL underperforms, the blended update provides a fallback that maintains low regret."],"supporting_citations":[{"why":"Defines the combinatorial DFL task and supplies the perturbation-based gradient estimator and budget-allocation problem used as a baseline and experiment.","marker":"Wilder, Dilkina, and Tambe 2019"},{"why":"Supplies differentiable convex optimization layers used to backpropagate through the solver for DFL gradients.","marker":"Agrawal et al. 2019"},{"why":"Provides the knapsack benchmark and dataset used in the unweighted and weighted knapsack experiments.","marker":"Mandi et al. 2020"},{"why":"Establishes task-based end-to-end learning through KKT-based implicit differentiation, a foundation for DFL training.","marker":"Donti, Amos, and Kolter 2017"},{"why":"Introduces a differentiable quadratic-programming optimization layer with KKT-based backpropagation for DFL gradient computation.","marker":"Amos and Kolter 2017"},{"why":"Documents the flat-and-sharp decision-loss landscape that motivates the paper's gradient-perturbation design.","marker":"Bansal et al. 2024"},{"why":"A gradient-surgery multi-task baseline compared in the experiments.","marker":"Yu et al. 2020"},{"why":"Multiple-gradient-descent baseline compared as a gradient-perturbation method.","marker":"Désidéri 2012"},{"why":"Dual-cone gradient-descent baseline compared as a gradient-perturbation method.","marker":"Hwang and Lim 2024"},{"why":"Supplies the portfolio-selection objective used in the portfolio optimization experiment.","marker":"Markowitz 2008"}],"fun_headline_variants":["Gradient blend tames DFL instability, cuts regret","Guide decision gradients with prediction loss, no extra cost","Geometric gradient mean stabilizes DFL, reaches Pareto point","Bisect decision and prediction gradients for lower regret"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The convergence guarantee assumes exact, differentiable decision-loss gradients, but the experiments that showcase the method—knapsack and budget allocation—have discrete decisions and use approximate surrogate gradients, so the theorem does not cover the settings where the method is demonstrated.","fun_headline_variants_meta":{"raw":{"variants":["Gradient blend tames DFL instability, cuts regret","Guide decision gradients with prediction loss, no extra cost","Geometric gradient mean stabilizes DFL, reaches Pareto point","Bisect decision and prediction gradients for lower regret"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1323,"prompt_tokens":743,"completion_tokens":580,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":514}},"tokens_in":487,"tokens_out":580,"duration_ms":6575,"temperature":1.0,"reasoning_tokens":514,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T20:39:02.874359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a smooth problem with exact decision gradients (portfolio optimization is one), run the kappa=0 update and record at every epoch the quantity the proof calls m_k psi_k: the combined progress toward both losses along the update direction. Theorem 3.4 predicts the minimum of this quantity over the first T epochs shrinks as O(T^-(1-alpha)) and tends to zero. If it stops decreasing while both individual gradient norms are clearly nonzero and their angle is below pi, the Pareto-stationary guarantee is falsified. A separate quick check: the paper's Proposition 3.2 says the merged update never ha","supporting_citations":[],"review_version":1}