{"id":"3b2cfa29-7321-4b97-a127-fb0604fb37dd","arxiv_id":"2502.01669","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"IF-DFM uses influence functions to update a conversion-rate model for delayed and newly arrived labels, approaching retraining accuracy at a fraction of the compute cost.","lead":"Online ad systems often wait days for a click to become a purchase, so freshly trained models can have wrong labels on recent events. This paper presents IF-DFM, which estimates how model parameters should shift when delayed conversions and new behaviors arrive, updating the model in about 15 seconds instead of retraining.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The unvalidated finite-epsilon influence approximation and the tacit assumption that Eq. (18) has a well-defined solution are load-bearing: if either fails, IF-DFM is not approximating retraining.","rationale":"The reader's weakest assumption is exactly the finite-epsilon validity of the influence approximation and the PSD Hessian condition, and the paper's own text supports this concern: the derivation in Eq. (8) is the standard infinitesimal result, Eq. (9) immediately substitutes ε = 1/n without an error bound, and Eq. (18) assumes only PSD while the preceding equations require invertibility. The related-work paragraph acknowledges Basu et al. (2021) on fragility but does not address it, and the limitations section mentions higher-order Taylor expansions as future work, which implicitly concedes that the first-order update is not fully justified. The experimental section does not report the parameter-space error between the IF update and an actual retrain, only downstream metrics, so the central mechanism is never directly validated. These are correctness risks rather than mere disagreements with prior consensus: they concern whether the update computed by Algorithm 1 is mathematically well-defined and close to the retrained solution. Because the paper is a proposal whose empirical evidence is promising but conditional on this validation, the reader's CONDITIONAL verdict is the appropriate one, and my read does not move it.","tokens_in":12956,"tokens_out":6238,"duration_ms":66717,"concrete_test":"On the Criteo MLP setting, fix interval [T, T′], identify label-reversal set J and new-data set K, and compute Δ_IF via Algorithm 1. Train the exact retrained model θ_retrain from the same θ̂ on corrected labels and compute Δ_retrain = θ_retrain − θ̂. Report ‖Δ_IF−Δ_retrain‖/‖Δ_retrain‖, AUC(θ̂+Δ_IF) versus AUC(θ_retrain), and ‖P_null b‖/‖b‖ for the Hessian at θ̂. If the relative parameter error is large, if the AUC gap is far from the claimed 85%, or if ‖P_null b‖/‖b‖ is non-negligible, the finite-epsilon approximation and well-posedness of Eq. (18) are not established. As a secondary check, rerun Algorithm 1 with ε reduced by 10× and 100×; the output should scale nearly linearly in ε if first-order influence is valid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of IF-DFM is Eq. (8)-(14): a first-order influence estimate is evaluated at ε = 1/n and then treated as the exact parameter change from finite label reversals and new samples. This step is load-bearing because every reported gain is attributed to the update Δθ_total, not to any auxiliary model or retraining. The paper cites Basu et al. (2021) showing that such influence estimates are fragile in deep networks, yet it provides no error bound, no comparison of Δθ_IF with Δθ_retrain, and no sensitivity study in ε. For deep, non-convex models, a full label flip is not an infinitesimal reweighting: the Hessian changes along the path from θ̂ to the retrained optimum, and first-order terms ignore interactions between the many simultaneously flipped and newly added samples. A second, independent problem is the linear algebra. Equations (8)-(14) require H^{-1}; the paper then replaces this by solving the PSD quadratic program (18). Positive semidefiniteness alone does not guarantee a finite minimizer: if b ∉ range(H), the objective is unbounded below along directions in null(H), so Algorithm 1 has no well-defined target. At a practical SGD solution of a deep network, H is typically singular and may have negative curvature, and the paper does not verify b ∈ range(H), add damping, or report the spectrum. Both issues concern the validity of the update mechanism itself, not just its accuracy, so the claim that IF-DFM 'approaches the performance of Retrain' is unsupported until they are checked.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IF-DFM, a delayed-feedback modeling method for CVR prediction. Instead of retraining when delayed conversions arrive, IF-DFM estimates the parameter change induced by (i) flipping fake-negative labels to positive and (ii) incorporating newly arrived behavioral data, using a first-order influence-function approximation. The inverse Hessian-vector product is reformulated as a finite-sum quadratic optimization problem and solved with ADAM. Experiments on Criteo and Taobao with four backbones compare IF-DFM against 13 baselines and report that IF-DFM significantly outperforms baselines and closes 85.16% of the AUC gap between Vanilla and Retrain.","tokens_in":13276,"tokens_out":5985,"duration_ms":61045,"significance":"If the central approximation is valid, this is a practically valuable contribution: it offers a way to update a deployed CVR model in about 15 seconds without retraining or auxiliary models. The paper provides experiments on two large-scale datasets across multiple backbones and baselines, and the algebraic reduction of the inverse Hessian-vector product to a finite-sum quadratic problem is a useful computational idea. However, the validity of the first-order influence approximation for finite label flips is not established, and the quadratic reformulation has a well-posedness gap when the Hessian is singular. The paper itself cites Basu et al. (2021) on the fragility of influence functions in deep learning but does not address that conflict. These issues are load-bearing because every reported gain is attributed to the approximate parameter update Δθ_total.","major_comments":[{"comment":"The core update rule is a first-order influence approximation evaluated at ε=1/n for finite label reversals and new-sample additions. The paper cites Basu et al. (2021) showing that influence functions in deep learning can be fragile, but it does not explain why the present setting is immune, nor does it provide an error bound or an empirical check. In particular, there is no comparison between Δθ_IF and the actual retrained parameter difference Δθ_retrain, and no sensitivity study in ε. Since all experimental gains are attributed to Δθ_total, this missing validation is load-bearing for the claim that IF-DFM 'approaches the performance of Retrain'.","section":"§Method, Eqs. (7)–(14)"},{"comment":"The assertion that positive semidefiniteness of the Hessian implies that Δθ_total is the solution of a convex quadratic program is not sufficient. If b ∉ range(H), the objective F(Δ) is unbounded below along directions in null(H), so Algorithm 1 has no well-defined target. The earlier influence-function derivation requires H to be invertible; replacing H^{-1} by a PSD quadratic solve does not remove the singularity issue. The authors should verify b ∈ range(H), add damping or regularization, solve a least-squares/min-norm formulation, or report relevant spectral information about the Hessian at the practical solution.","section":"§An Efficient and Scalable Method, Eq. (18)"},{"comment":"The evaluation reports only point estimates. The double asterisks indicate a t-test with p≤0.05, but the number of independent runs, standard deviations, and confidence intervals are not reported, making the significance claim unverifiable. Please report these statistics for the main comparisons, including the relative-improvement numbers that support the central claim that IF-DFM closes most of the gap to Retrain.","section":"§Experiment, Tables 2–3 and Figures 3–4"},{"comment":"The new-data term in Eq. (14) includes newly arrived samples z_k with their observed labels as if those labels were ground truth. However, a click observed in [T,T′] without a conversion by T′ is itself a potential fake negative because conversions may be delayed beyond T′. The paper does not model or correct this, and the definition of the new-data perturbation should be stated with the same label-reversal logic used for old samples, or the assumption should be justified explicitly.","section":"§Method, Eq. (14) and online setting"}],"minor_comments":[{"comment":"There are multiple typos and formatting issues, including 'Auxiliay Model' in Figure 1, 'i nfluence f unction' in the introduction, 'approximate∆θtotal' missing a space, and 'to some extend' in the related work.","section":"Throughout"},{"comment":"The text says runtime results are shown in Figure 4, but the actual runtime table is Table 4. Also clarify what the 'Vanilla + Ours' time includes (e.g., whether the 14.8 seconds includes data preprocessing and constructing b).","section":"§Efficiency of Calculating Parameter Changes"},{"comment":"The manuscript repeatedly refers to an Appendix for baseline descriptions, implementation details, additional results, and the derivation of Eq. (11), but no appendix is present in the submitted version. Either include the appendix or remove the references.","section":"References and Appendix"},{"comment":"The Nocedal and Wright reference is incomplete: the year and publisher are listed as '????'.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and practically important problem, and the computational reformulation is attractive. However, the central approximation is not validated, and the quadratic reformulation has a well-posedness gap that affects the correctness of Algorithm 1. The requested revisions—direct validation of Δθ_IF against retraining, handling of singular Hessians, and statistical reporting—are within the scope of the manuscript and should be feasible without changing the overall approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper: it applies influence functions to delayed-feedback CVR prediction, treating label reversals and new data as perturbations, and replaces the inverse Hessian-vector product with a finite-sum quadratic program solved by ADAM. The framing is new, the math in Eqs (7)-(20) is algebraically consistent, and the experiments show consistent gains across four backbones, with runtime of ~15 seconds versus retraining. That part is real. It is a solid subfield contribution, not a field-reshaping one.\n\nWhat the paper does well: the reformulation of the IHVP as an optimization problem is a genuine practical move, and the empirical gains, while small in absolute terms, are consistent and economically relevant for ad platforms. The relative improvement over Retrain (85-86% of the AUC gap on Criteo) is striking, and the ablation with IF-DFM-w/o-add shows the new-data term helps.\n\nNow the soft spots, and they are load-bearing. The first-order influence approximation is evaluated at epsilon = 1/n for full label flips in deep networks. The paper cites Basu et al. 2021 showing influence functions in deep learning are fragile, but gives no error bound, no comparison of the predicted parameter change against the actual retrained model, and no sensitivity study in epsilon. That is not a minor omission; the entire claim that IF-DFM approximates Retrain rests on this step. Second, the quadratic program in Eq (18) assumes the Hessian is positive semidefinite. The paper argues this follows from second-order necessary conditions, but at a practical SGD solution of a deep network the Hessian is typically singular and may have negative curvature; if b is not in the range of H, the objective is unbounded below and ADAM has no well-defined target. The paper does not check this, add damping, or report the spectrum. The nnDF baseline also performs implausibly poorly (AUC around 0.68 vs 0.84 for Vanilla), which suggests a broken or mismatched implementation; that weakens the baseline comparison but does not by itself invalidate the method.\n\nMy take: the central idea is sound and worth pursuing, but the evidence as presented does not fully support the strength of the claims. The paper deserves a serious referee, meaning a good venue should send it out and ask for the finite-epsilon validation and Hessian diagnostics. If those check out, this is a useful contribution.\n\nRecommendation: send to peer review, with heavy revision pressure.","headline":"A plausible, clearly derived application of influence functions to delayed-feedback CVR, with a correct-looking quadratic reformulation, but the load-bearing finite-epsilon approximation and the Hessian assumptions are not validated.","tokens_in":13799,"tokens_out":829,"would_cite":true,"duration_ms":9856,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Influence functions can update a deployed conversion-rate model in about 15 seconds, absorbing delayed conversions and new behavior data without retraining.","keywords":["delayed feedback","conversion rate prediction","CVR","influence functions","inverse Hessian-vector product","online advertising","label reversal","stochastic optimization"],"falsifier":"Retrain a backbone (for example, MLP) on the Criteo data with corrected labels for a fixed delay window, then compare the retrained model's test AUC and parameter vector with those of the IF-DFM-updated vanilla model; if they are not close (the AUC gap is not roughly 85 percent of the vanilla-retrain gap, or the parameter vectors differ greatly), the first-order influence approximation is not the mechanism behind the reported gains.","tokens_in":12754,"feed_emoji":"⚡","tokens_out":13956,"duration_ms":110686,"temperature":0.7,"pith_summary":"IF-DFM claims that a deployed conversion-rate (CVR) model can absorb delayed conversions and newly arrived click data without retraining, by using influence functions to estimate how model parameters would change under label corrections. The paper treats a fake negative turning into a true positive as a sample reweighting, and the arrival of new data as another reweighting, then approximates the combined parameter update by a single inverse-Hessian-vector product. To make that computation tractable, it reformulates the product as a finite-sum convex quadratic program and solves it with Adam, taking about 15 seconds on a Criteo-scale model. Experiments across four backbones and two public datasets report that this update significantly outperforms existing offline and online delayed-feedback baselines and approaches the accuracy of full retraining, including 85.16% of the AUC gap between vanilla and retrain on Criteo. If correct, the method offers a practical alternative to auxiliary delay models and sample duplication for keeping CVR models fresh.","feed_headline":"A 15-second model update captures 85% of retraining gains","feed_subtitle":"Influence functions let a deployed CVR model absorb late conversions and new clicks in seconds, near retrain-quality accuracy.","key_machinery":"The load-bearing object is the influence function of the vanilla-trained model, evaluated at the empirical minimizer $\\hat\\theta$, whose parameter-change formula $\\Delta\\theta \\approx -\\epsilon H_{\\hat\\theta}^{-1}\\nabla_\\theta L(z,\\hat\\theta)$ is the standard result from the influence-function literature. The paper's specific machinery is the reformulation of the required inverse-Hessian-vector product as the solution of the finite-sum quadratic program (Eqs. 18-20), where each summand uses the per-sample Hessian $\\nabla_\\theta^2 L_{\\text{BCE}}(z_i,\\hat\\theta)$ and a shared vector $b$ built from gradients of label-reversed and newly added samples. Because $\\hat\\theta$ is a minimizer of the vanilla loss, the Hessian is positive semidefinite, making the quadratic program convex and solvable by stochastic optimizers such as Adam, with auto-differentiation and Hessian-vector products supplying each $f_i$. This replaces direct Hessian inversion with a scalable optimization problem, which is what makes the delayed-feedback update practical.","core_discovery":"IF-DFM's central claim is that the effect of delayed feedback on a trained CVR model can be captured, to a good approximation, by a first-order influence function acting on the vanilla loss. A label reversal from 0 to 1 for a sample $z_i$ is represented as a perturbation with $\\delta=1$, and a newly arrived sample $z$ is integrated with effective weight $\\epsilon=1/(n+1)$; both are combined into a single parameter displacement $\\Delta\\theta_{\\text{total}} = \\Delta\\theta_{\\text{delay}} + \\Delta\\theta_{\\text{add}}$ given by the inverse Hessian of the vanilla loss times a sum of gradient differences (Eq. 14). The paper then converts this inverse-Hessian-vector product into the solution of the convex quadratic program $\\min_{\\Delta} F(\\Delta) = \\frac{1}{2}\\Delta^\\top \\nabla_\\theta^2 L_V(\\hat\\theta)\\Delta - \\langle b, \\Delta\\rangle$, whose objective has a finite-sum structure, enabling scalable solution by Adam. On the Criteo and Taobao datasets, with MLP, DeepFM, AutoInt, and DCNV2 backbones, the updated model is reported to outperform thirteen baselines across AUC, PRAUC, and log loss, close about 85 percent of the vanilla-to-retrain AUC gap, and do so in about 14.8 seconds of additional computation.","pith_inferences":["The paper does not directly compare the influence-updated parameters to the parameters produced by actual retraining; measuring that distance (for example, cosine similarity or prediction alignment) would test whether the reported gains come from the approximation or from the small update step itself.","The first-order influence formula is exact only for infinitesimal reweighting, and the paper itself cites work showing influence functions can be fragile in deep learning; applying it at $\\epsilon=1/n$ for full label flips deserves an error bound or a perturbation test that the paper does not provide.","The finite-sum quadratic reformulation is essentially a scalable iterative solver for inverse-Hessian-vector products, so the same trick could be reused in other influence-function applications, such as data debugging, unlearning, or explainability, where full Hessian inversion is currently a bottleneck.","The reported 14.8-second update time and the approximation quality likely depend on the number of label-reversed and newly arrived samples and on the Hessian's condition number; a reader should expect these to trade off when the method is scaled to much larger or more non-convex models."],"forward_implications":["A deployed CVR model can be refreshed within seconds after each delay window by a direct parameter update, rather than waiting for a full retraining cycle.","The approach removes the need for auxiliary delay-distribution models, label-correction networks, or duplicated fake-negative samples, simplifying the training pipeline.","The same update mechanism works in offline and online settings and across MLP, DeepFM, AutoInt, and DCNV2 backbones, so it can be applied to existing deployed models.","Because the update closes roughly 85 percent of the vanilla-to-retrain metric gap, models can track evolving user interest with less label staleness between retraining periods."],"supporting_citations":[{"why":"Supplies the influence-function parameter-change formula and the inverse-Hessian-vector product that the method builds on.","marker":"(Koh and Liang 2017)"},{"why":"Defines the delayed feedback problem in display advertising and provides the DFM baseline and vanilla-retrain framing.","marker":"(Chapelle 2014)"},{"why":"Introduces the influence function in robust statistics, the conceptual foundation of the parameter-update estimate.","marker":"(Hampel 1974)"},{"why":"Provides the FSIW baseline with feedback-shift correction that IF-DFM compares against and improves upon.","marker":"(Yasui et al. 2020)"},{"why":"Provides the FNW/FNC online baselines for continuous training with delayed feedback used in the experiments.","marker":"(Ktena et al. 2019)"},{"why":"Provides the ULC baseline and the temporal partitioning protocol for dividing the datasets.","marker":"(Wang et al. 2023)"},{"why":"Supplies the Adam optimizer used to solve the finite-sum quadratic program for the parameter update.","marker":"(Kingma and Ba 2015)"},{"why":"Supplies the second-order optimality condition and conjugate-gradient background that justify the quadratic reformulation.","marker":"(Nocedal and Wright)"}],"fun_headline_variants":["Influence functions close 85% of retraining gap in 15s","IF-DFM: 15s update captures 85% of retraining AUC gain","15-second influence-function update gives 85% of full retraining","Influence functions: near-retrain CVR accuracy in 15 seconds, no full retraining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a first-order influence approximation, which is exact only for tiny data changes, stays accurate when applied to full label corrections and new samples at realistic sample sizes, so the computed parameter update actually matches what retraining would produce.","fun_headline_variants_meta":{"raw":{"variants":["Influence functions close 85% of retraining gap in 15s","IF-DFM: 15s update captures 85% of retraining AUC gain","15-second influence-function update gives 85% of full retraining","Influence functions: near-retrain CVR accuracy in 15 seconds, no full retraining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001442,"raw_usage":{"total_tokens":5836,"prompt_tokens":996,"completion_tokens":4840,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":4752}},"tokens_in":612,"tokens_out":4840,"duration_ms":29682,"temperature":1.0,"reasoning_tokens":4752,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:48:01.319691+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain a backbone (for example, MLP) on the Criteo data with corrected labels for a fixed delay window, then compare the retrained model's test AUC and parameter vector with those of the IF-DFM-updated vanilla model; if they are not close (the AUC gap is not roughly 85 percent of the vanilla-retrain gap, or the parameter vectors differ greatly), the first-order influence approximation is not the mechanism behind the reported gains.","supporting_citations":[],"review_version":1}