Influence functions and regularity tangents for efficient active learning

Frederik Eaton

REVIEW 3 major objections 4 minor 24 references

Influence functions and regularity tangents for efficient active learning

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a single 'regularity tangent' vector, computed during training with constant slowdown, turns any candidate data point's loss gradient into an influence function on model complexity, and that squaring this score…

desk verdict Clever identity and a nice algorithmic observation, but the active learning payoff is entirely untested; worth a referee's time only if the empirical gap is filled. read the letter →

arxiv 2411.15292 v2 pith:ZLLABGMW submitted 2024-11-22 cs.LG cs.AImath.STstat.MLstat.TH

Frederik Eaton This is my paper · ORCID

classification cs.LGcs.AImath.STstat.MLstat.TH

keywords activelearninginfluencefunctionsregularitytangentregularizedregressionstochasticgradientdescentforward-modeautomaticdifferentiationLiSSAqueryheuristics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes an active-learning rule for regression models whose query-time cost is essentially zero. The author introduces a 'regularity tangent'—the derivative of the optimal parameter vector with respect to the regularization strength—and shows it can be computed alongside the parameters during stochastic gradient descent with only a constant slowdown and one extra stored vector. Once training is finished, scoring a candidate unlabeled point is an inner product of this tangent with the point's loss gradient, so it costs the same as computing that gradient. The paper proves that this score is an influence function measuring the expected squared change in model complexity (the regularizer) that would result from adding the point, and it proposes several query heuristics built on that quantity. The practical payoff, if the paper's guiding hypothesis is right, is that large models could choose which labels to request without retraining or Hessian inversion.

What carries the argument

The load-bearing object is the regularity tangent $\frac{d\theta^*}{ds} = -H^{-1}\rho$, the derivative of the optimal parameter vector with respect to the regularity hyperparameter $s$, where $H$ is the Hessian of the training objective and $\rho = \frac{\partial^2 R}{\partial \theta \partial s}$ is the complexity gradient (for L2 regularization, $\rho = 2\theta^*$). The companion identity $\frac{dL(z,\theta^*)}{ds} = \frac{dR_s(\theta^*)}{d\varepsilon}$ turns this tangent into an influence function on model complexity, so that the inner product with a point's loss gradient and the squaring of that scalar become the query score. The computational mechanism is SGDF: running SGD on dual numbers adds the update $\dot{\theta}_{t+1} \leftarrow \dot{\theta}_t - \eta_t(\rho + H\dot{\theta}_t)$ to the ordinary parameter update, so $\dot{\theta}$ converges to $d\theta^*/ds$ with the same step-size schedule as $\theta$ and without Hessian inversion.

What would settle it

Falsify the recommendation by computing, on a regression benchmark, the squared-loss-derivative score for every unlabeled pool point, then adding labels in descending score order: if the cumulative held-out error after each query is not better than random label selection (or than a residual-based baseline), the Section 3.1 Hypothesis is false.

Watch

Extended reading notes

Core claim

The central claim is the identity at an optimum $\theta^*$ of any smooth regularized objective: $\frac{dL(z,\theta^*)}{ds} = \frac{dR_s(\theta^*)}{d\varepsilon}$, where the left side is the derivative of a point's loss with respect to the regularity hyperparameter $s$ and the right side is the derivative of the regularizer with respect to an infinitesimal up-weighting $\varepsilon$ of that point. Because $\frac{d\theta^*}{ds} = -H^{-1}\rho$, with $\rho$ the complexity gradient of the regularizer, the left side is a single vector-inverse-Hessian-vector product $-\sigma_z^\top H^{-1}\rho$. The paper takes the squared value of this scalar, averaged over plausible labels, as a query heuristic that prefers the point whose inclusion would produce the greatest expected squared change in model complexity. It then presents SGDF, a dual-number version of SGD whose updates carry $\dot{\theta} = d\theta/ds$ alongside $\theta$ and converge to $d\theta^*/ds$ without ever forming $H^{-1}$; the LiSSA algorithm is shown to be a special case of SGDF when the regularizer update is applied at every stochastic step. An illustrative polynomial-regression experiment shows the resulting score coincides with the summed influence of all training points, a direct corollary of stationarity at the optimum.

Load-bearing premise

The load-bearing premise is the Section 3.1 Hypothesis that the data points most able to stabilize a model are precisely those whose predictions vary most when the regularity hyperparameter is varied; if that premise is false, the squared-loss-derivative score could be no better than random label selection.

Editorial extensions

If this is right

Once training is complete, scoring a candidate data point costs the same as computing its loss gradient, since only the inner product with the precomputed regularity tangent is needed.
Only one extra vector of the same dimension as the parameters is stored, so the method approximately doubles memory and adds a constant factor to training time.
The query heuristic selects points whose labels are expected to cause the largest squared change in model complexity, which is a different objective from maximizing prediction uncertainty or test-loss reduction.
SGDF produces the regularity tangent during ordinary training, making it possible to update the regularity hyperparameter $s$ stochastically from a held-out pool at the same time.
In a multi-user model with a shared base parameter vector, each user's regularity tangent inherits from the common tangent, so even a brand-new user can receive a non-random first query.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural untested extension is to use the squared-loss-derivative score as the ranking function in batch or pool-based active learning, taking the top-$k$ scores each round; the paper's cost analysis suggests this would remain cheap.
The duality $dL/ds = dR/d\varepsilon$ is more general than active learning: the same computation could be used as a data-curation diagnostic to identify which training points most affect model complexity, even when no new labels are being purchased.
Because SGDF inherits its step size from the underlying SGD schedule, it may resolve the tuning and poor-convergence issues reported for LiSSA on ill-conditioned problems; a direct comparison on a badly conditioned objective would test that.
The hierarchical regularizer models suggest a testable path to personalized large models: each user's tangent can couple sparse per-user gradients through the inverse Hessian, so the method's usefulness in federated settings could be measured without sharing raw data.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Frederik Eaton's regularity tangent paper is a mixed bag: the math at the center is real, and the algorithmic observation is neat, but the paper sells itself as an active learning method and the evidence for that is absent.

The genuinely new thing is the identity dL(z,θ*)/ds = dR_s(θ*)/dε (Eqs. 68-70). It follows from the implicit function theorem and symmetry of the Hessian, and it gives a way to compute a data point's influence on model complexity with a single inner product after training. That's a real contribution. The SGDF algorithm — dual-number SGD that propagates dθ/ds while training — is also a reasonable forward-mode extension of SGD, and the observation that LiSSA is a special case of it (Section 3.6) is nice. The paper's literature review is honest and it's self-contained. Credit is due for the derivation.

The soft spots are exactly where the reader put them. The active learning claim rests entirely on the 'Hypothesis' in §3.1: points whose predictions vary most with regularity s are the ones best able to stabilize the model. No derivation connects |dL/ds| to expected reduction in generalization error or label informativeness. The paper's own §3.3 defers all comparisons, and §3.2's body is removed. There is no query loop, no test-set evaluation, no code. The plots are illustrations, not evidence. The SGDF dual iterate also has no convergence proof — the paper says 'if it converges' — so the constant-slowdown claim is conditional.

None of this is a demonstrated mathematical error; the identity is sound. But the central claim of an efficient active learning method is untested, and the paper openly admits it. A benchmark comparing the squared-loss-derivative query rule to random selection on a simple regression problem would settle the question, and it's missing.

Who should read this? Anyone working on influence-based active learning or efficient Hessian-free influence computation will want to know about the identity and SGDF/LiSSA equivalence. But treat it as a theoretical note, not a method paper. It deserves a serious referee — the math is worth checking — but only if the referee insists on at least one experiment before acceptance. I'd send it to review, with the expectation of heavy revision or a demotion to a theory-only contribution.

Referee Report

3 major / 4 minor

Summary. The paper proposes an active-learning query heuristic for regularized regression based on a "regularity tangent" dθ*/ds = -H^{-1}ρ, where s is a scalar regularity hyperparameter and ρ = ∂²f/∂θ∂s. The central identity (Eqs. 68-70) shows that the loss derivative dL(z,θ*)/ds equals the influence of up-weighting a candidate point z on the regularizer derivative dR_s(θ*)/dε via Hessian symmetry. Squaring this scalar and averaging over plausible labels gives the squared loss derivative (SLD) query heuristic. The paper then proposes SGDF (Eqs. 117-125), a forward-mode automatic-differentiation extension of SGD that computes dθ*/ds alongside training with constant slowdown, and claims that SGDF generalizes the LiSSA algorithm. Additional sections sketch multi-user regularizer hierarchies, SGDF-based hyperparameter optimization, and second-order variants. The manuscript contains no query-loop experiments or test-set evaluations; Section 3.2, which was to describe additional heuristics, is removed, and Section 3.3 explicitly defers empirical comparisons.

Significance. If the main identity and the SGDF complexity claims hold, the paper contributes a cheap influence-type query score for large regression models and a useful formal connection between SGDF and LiSSA. The derivation of the identity is self-contained and follows from the implicit function theorem and symmetry of the Hessian, and the SGDF update is a coherent dual-number version of SGD; no constants are fitted to make the mathematics agree with data. However, the practical value of the SLD heuristic is entirely contingent on the unvalidated Hypothesis in Section 3.1, and the absence of any empirical evaluation means the central active-learning claim is not yet established.

major comments (3)

[§3.1, §3.3] The active-learning claim rests on the untested Hypothesis in Section 3.1: points whose model predictions vary most with regularity s are those best able to stabilize the model. No derivation connects |dL(z,θ*)/ds|² to expected reduction in generalization error, label informativeness, or any downstream objective, and Section 3.3 explicitly postpones comparisons with other query heuristics and evaluation on real data. Since the score could select high-leverage or outlier points without improving predictions, the manuscript needs at least a query-loop benchmark comparing SLD with random acquisition and standard uncertainty-based heuristics on the polynomial example and one or two real regression datasets; as written, the central claim is unvalidated.
[§3.5, Eqs. (114), (120), (125)] The regularity tangent estimate requires the dual iterates ˙θ_t to converge to -H^{-1}ρ, but the text only says "If it converges" and gives no convergence conditions or proof. For stochastic updates with a step-size schedule and adaptive methods such as Adam, the claimed constant-slowdown estimate of -H^{-1}ρ is not guaranteed even when θ_t converges. The authors should either prove convergence under stated assumptions, such as convex quadratic losses with decaying step sizes, or provide empirical convergence diagnostics showing that ˙θ_t tracks the exact regularity tangent on the example problem.
[§3.2] The section describing additional regularity-tangent derived query heuristics has been removed from the draft, yet the abstract and conclusion refer to "a number of ways" of using the proposed quantities. Without the contents of Section 3.2, the set of proposed heuristics is incomplete and cannot be evaluated. The authors should either restore the section or explicitly state that only the SLD heuristic (together with the influence heuristics of Section 2.4.2) is proposed; the current pointer to a missing section is not a substitute.

minor comments (4)

[§2.4.2, Eq. (61)] In Eq. (61), the notation "0 = df/dθ|_{θ∗=0}" is confusing; the derivative should be evaluated at the optimum θ = θ*, not at θ* = 0.
[§2.4, Eq. (38)] The last expression in Eq. (38) omits the argument θ* in the loss gradient; it should read -∂L/∂θ(z_test,θ*)⊤H^{-1}∂L/∂θ(z,θ*).
[§3.3] The notation in Eq. (77) is ambiguous: power(x) is first described as a 6-element vector for scalar x and then as a 6-column matrix for vector x; using different symbols for the two cases would improve clarity.
[§3.7] The proposal for joint stochastic optimization of θ and s is explicitly untested in the text and should be clearly labeled as speculative rather than as a demonstrated contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the influence identity is derived from Hessian symmetry, and the active-learning heuristic is explicitly an untested hypothesis, not a fitted or self-cited conclusion.

full rationale

The derivation chain is self-contained. Equations (64)-(70) define the regularity tangent as dθ*/ds = -H^{-1}ρ and prove dL(z,θ*)/ds = -σ_z^T H^{-1}ρ = -ρ^T H^{-1}σ_z = dR_s(θ*)/dε using only the implicit function theorem, the symmetry of the Hessian, and the chain rule; none of these steps assumes the active-learning outcome. SGDF in (112)-(125) is a direct forward-mode AD instrumented version of the SGD updates, and the claimed equivalence to LiSSA in §3.6 is shown by algebraic rearrangement of the LiSSA recursion. No parameter is fitted to data and then reported as a prediction, no load-bearing self-citation appears, and no prior result by the author is invoked to force a choice. The active-learning claim does rest on the explicit 'Hypothesis' in §3.1 that points with greatest variation under regularity changes are those best able to stabilize the model, and §3.3 candidly postpones empirical comparison ('future versions of this section should contain a comparison of the various query heuristics... time constraints force us to postpone such experiments for later'). That is an empirical validity gap, not circular reasoning: the heuristic is presented as a hypothesis, not derived from the identity by construction. The closest structural concern—that the squared-loss-derivative score is justified by the same intuition it is meant to instantiate—does not involve fitting or definitional equivalence and therefore is not a circular step under the criteria here.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central identity is derived from standard calculus and needs no fitted constants. The load-bearing assumptions are the unproven convergence of SGDF and an untested hypothesis connecting regularity sensitivity to active learning value. The regularizer R and the hyperparameter s are modeling choices, not data-fitted parameters, although s and the step schedule are free in practice.

free parameters (2)

regularity hyperparameter s = In the illustrative example it is 0.0344, chosen by LOOCV; otherwise it is user-selected.
The regularity tangent is defined as dθ*/ds, and the resulting query score depends on the chosen s. The paper provides no automatic rule for s beyond standard cross-validation.
SGD step-size schedule η_t = Not specified in the paper.
SGDF inherits the step-size schedule of the underlying SGD; the paper does not specify a schedule or convergence conditions for the dual iterate.

assumptions (5)

standard math The loss and regularizer are smooth enough and the Hessian H is invertible at θ*, so the implicit function theorem applies (Section 2.4, Eqs. 32-34).
Every influence function, including the regularity tangent, is derived from the stationary condition ∂f/∂θ=0 and requires H^{-1} to exist.
domain assumption The SGDF dual-number updates converge to the fixed point H ˙θ = -ρ (Section 3.5).
This convergence is not proved; the text only says "If it converges, from (114) we can see that it should converge". Without it, the constant-slowdown computation of dθ*/ds is not guaranteed.
ad hoc to paper Active learning hypothesis: points whose model predictions vary most with regularity s are the most valuable to label (Section 3.1).
Labeled explicitly as a "Hypothesis" and never tested; it is the entire justification for the squared loss derivative query score.
domain assumption For squared loss, the label model p_y(y|x,θ) is Gaussian centered at θᵀx (footnote 5, Eqs. 48-54).
Used to compute the expectation Q(x)=E[Q(z)] in the squared loss derivative and STI/SSI heuristics.
ad hoc to paper A single scalar regularity s suffices to rank model complexity (Section 3.7).
The paper's final generality argument that "all models may be ranked by complexity" is a philosophical assumption needed to extend the method beyond explicit regularizers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Influence functions and regularity tangents for efficient active learning." pith.science (2026). https://pith.science/paper/ZLLABGMW

@misc{pith2026241115292,
  author       = {Pith},
  title        = {Pith review of: Influence functions and regularity tangents for efficient active learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZLLABGMW}},
  note         = {Machine review of arXiv:2411.15292}
}

read the original abstract

In this paper we describe an efficient method for providing a regression model with a sense of curiosity about its data. In the field of machine learning, our framework for representing curiosity is called Active Learning, which concerns the problem of automatically choosing data points for which to query labels in the semi-supervised setting. The methods we propose are based on computing a "regularity tangent" vector that can be calculated (with only a constant slow-down) together with the model's parameter vector during training. We then take the inner product of this tangent vector with the gradient vector of the model's loss at a given data point to obtain a measure of the influence of that point on the complexity of the model. In the simplest instantiation, there is only a single regularity tangent vector, of the same dimension as the parameter vector. Thus, in the proposed technique, once training is complete, evaluating our "curiosity" about a potential query data point can be done as quickly as calculating the model's loss gradient at that point. The new vector only doubles the amount of storage required by the model. We show that the quantity computed by our technique is an example of an "influence function", and that it measures the expected squared change in model complexity incurred by up-weighting a given data point. We propose a number of ways for using this and other related quantities to choose new training data points for a regression model.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages

[1]

Second-Order Stochastic Optimization for Machine Learning in Linear Time, 2017

Agarwal, Bullins, and Hazan. Second-Order Stochastic Optimization for Machine Learning in Linear Time, 2017

work page 2017
[2]

Understanding Black-box Predictions via Influence Functions, 2017

Koh and Liang. Understanding Black-box Predictions via Influence Functions, 2017

work page 2017
[3]

Automatic differentiation and iterative processes, 1992

Gilbert. Automatic differentiation and iterative processes, 1992

work page 1992
[4]

Gradient-Based Optimization of Hyper-Parameters, 1999

Bengio. Gradient-Based Optimization of Hyper-Parameters, 1999

work page 1999
[5]

Gradient-based Hyperparameter Optimization through Reversible Learning, 2015

Maclaurin, Duvenaud, and Adams. Gradient-based Hyperparameter Optimization through Reversible Learning, 2015

work page 2015
[6]

Forward and Reverse Gradient-Based Hy- perparameter Optimization, 2017

Franceschi, Donini, Frasconi, and Pontil. Forward and Reverse Gradient-Based Hy- perparameter Optimization, 2017

work page 2017
[7]

Bilevel Programming for Hyperparameter Optimization and Meta-Learning, 2018

Franceschi, Frasconi, Grazzi, and Pontil. Bilevel Programming for Hyperparameter Optimization and Meta-Learning, 2018

work page 2018
[8]

Optimizing Millions of Hyperparameters by Implicit Differentiation, 2020

Lorraine, Vicol, and Duvenaud. Optimizing Millions of Hyperparameters by Implicit Differentiation, 2020

work page 2020

Show all 24 references

[9]

Residuals and Influence in Regression, 1982

Cook and Weisberg. Residuals and Influence in Regression, 1982

1982
[10]

Influence Selection for Active Learning, 2021

Liu. Influence Selection for Active Learning, 2021

2021
[11]

Scaling Up Influence Functions, 2021

Schioppa et al. Scaling Up Influence Functions, 2021

2021
[12]

Reliable Active Learning via Influence Functions, 2023

Xia et al. Reliable Active Learning via Influence Functions, 2023

2023
[13]

Maximizing Expected Model Change for Active Learning in Regression, 2013

Cai, Zhang, and Zhou. Maximizing Expected Model Change for Active Learning in Regression, 2013. 36

2013
[14]

A Swiss Army Infinitesimal Jackknife, 2019

Giordano et al. A Swiss Army Infinitesimal Jackknife, 2019

2019
[15]

Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter, 1979

Golub et al. Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter, 1979

1979
[16]

Active Learning Literature Survey, 2010

Settles. Active Learning Literature Survey, 2010

2010
[17]

Improving Generalization with Active Learning, 1994

Cohn, Atlas, and Ladner. Improving Generalization with Active Learning, 1994

1994
[18]

Adam: A Method for Stochastic Optimization, 2015

Kingma and Ba. Adam: A Method for Stochastic Optimization, 2015

2015
[19]

Practical Recommendations for Gradient-Based Training of Deep Architec- tures, 2012

Bengio. Practical Recommendations for Gradient-Based Training of Deep Architec- tures, 2012

2012
[20]

Generic Methods for Optimization-Based Modeling, 2012

Domke. Generic Methods for Optimization-Based Modeling, 2012

2012
[21]

Fast Exact Multiplication by the Hessian, 1993

Pearlmutter. Fast Exact Multiplication by the Hessian, 1993

1993
[22]

Automatic Differentiation in Machine Learning: a Survey, 2018

Baydin et al. Automatic Differentiation in Machine Learning: a Survey, 2018

2018
[23]

Gradients without Backpropagation, 2022

Baydin et al. Gradients without Backpropagation, 2022

2022
[24]

An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, 1994

Shewchuk. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, 1994. 37

1994

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Second-Order Stochastic Optimization for Machine Learning in Linear Time, 2017

Agarwal, Bullins, and Hazan. Second-Order Stochastic Optimization for Machine Learning in Linear Time, 2017

work page 2017

[2] [2]

Understanding Black-box Predictions via Influence Functions, 2017

Koh and Liang. Understanding Black-box Predictions via Influence Functions, 2017

work page 2017

[3] [3]

Automatic differentiation and iterative processes, 1992

Gilbert. Automatic differentiation and iterative processes, 1992

work page 1992

[4] [4]

Gradient-Based Optimization of Hyper-Parameters, 1999

Bengio. Gradient-Based Optimization of Hyper-Parameters, 1999

work page 1999

[5] [5]

Gradient-based Hyperparameter Optimization through Reversible Learning, 2015

Maclaurin, Duvenaud, and Adams. Gradient-based Hyperparameter Optimization through Reversible Learning, 2015

work page 2015

[6] [6]

Forward and Reverse Gradient-Based Hy- perparameter Optimization, 2017

Franceschi, Donini, Frasconi, and Pontil. Forward and Reverse Gradient-Based Hy- perparameter Optimization, 2017

work page 2017

[7] [7]

Bilevel Programming for Hyperparameter Optimization and Meta-Learning, 2018

Franceschi, Frasconi, Grazzi, and Pontil. Bilevel Programming for Hyperparameter Optimization and Meta-Learning, 2018

work page 2018

[8] [8]

Optimizing Millions of Hyperparameters by Implicit Differentiation, 2020

Lorraine, Vicol, and Duvenaud. Optimizing Millions of Hyperparameters by Implicit Differentiation, 2020

work page 2020