AD-HOC: A C++ Expression Template package for high-order derivatives backpropagation

Juan Lucas Rey

REVIEW 4 major objections 5 minor 15 references

AD-HOC: A C++ Expression Template package for high-order derivatives backpropagation

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AD-HOC is a C++ automatic differentiation library that computes user-specified high-order derivatives in a single backpropagation pass, with runtimes close to handwritten code.

desk verdict A real and clearly described C++ AD library with a novel compile-time design, but the headline performance ratio rests on single-run timings and a table that contradicts its own prose. read the letter →

arxiv 2412.05300 v2 pith:MUN3WQ6V submitted 2024-11-25 cs.MS q-fin.CP

Juan Lucas Rey This is my paper · ORCID

classification cs.MSq-fin.CP

keywords automaticdifferentiationhigh-orderderivativesexpressiontemplatesTaylorbackpropagationcompile-timecomputationtreeBlack-ScholesVanna-VolgaC++templatemetaprogramming

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AD-HOC is a C++ automatic differentiation library that computes user-selected derivatives of any order in a single reverse pass. Its central claim is that by encoding the entire calculation as a compile-time type, with each input distinguished by a unique type and constants stored as template parameters, it can match the runtime of handwritten derivative code. The paper demonstrates this on the Black-Scholes formula, where computing price, Vega, Vanna, and Volga with AD-HOCv2 takes 44 microseconds per million evaluations versus 46 for purpose-built handwritten code. For a general four-input tensor up to order 5, AD-HOC's runtime ratio is 16.66x, compared to 634x for the dynamic ADOL-C tool. A sympathetic reader would take this as evidence that static, type-based AD can deliver both high-order derivatives and practical speed without source-code generation.

What carries the argument

The central object is the compile-time calculation tree built from expression templates: a C++ type that contains zero data yet fully describes the computation, including constants encoded as template parameter values. The pruning mechanism is the BackPropagator, which takes only the requested derivative seeds and runs truncated Taylor expansions through each node, truncating monomials that do not contribute to the requested derivatives. Memory is handled by a CalcTree that allocates a single stack std::array sized at compile time, plus a buffer sized to the maximum backpropagation slice. Because the tree is fully static, the compiler can omit storing intermediate values that backpropagation does not need, which the paper credits for the runtime advantage.

What would settle it

Run AD-HOC on a large calculation tape, such as a portfolio of hundreds of Black-Scholes options, request order-5 tensors, and measure both compile time and the runtime ratio R(5). A compile time that grows superlinearly with the number of nodes, or an R(5) far above the reported 16.66x, would falsify the claim that AD-HOC keeps handwritten-code-level performance at scale.

Watch

Extended reading notes

Core claim

The paper's central claim is that high-order derivatives can be computed efficiently by representing the full calculation tree as a data-less C++ type and then performing a Taylor-expansion backpropagation over that static tree. In AD-HOC, each input variable has a unique type, constants are encoded as template parameters, and expressions build nested types such as mul_t<double_t<"S">, erfc_t<...>> that describe the whole computation with zero runtime data. The backpropagator receives only the derivative seeds the user needs, such as d(V)*d(S) or d<2>(V), propagates truncated Taylor series through each node by substituting one series into another, and multiplies the resulting coefficients by factorials to obtain the requested derivatives in a single tree pass. The paper reports that this yields Black-Scholes price plus three Greek sensitivities at a runtime 1.19x to 1.45x the plain price evaluation, essentially matching handwritten code, and that a four-input full tensor to order 5 costs 16.66x the primal evaluation versus 634x for ADOL-C.

Load-bearing premise

The load-bearing premise is that the paper's measured performance on a small Black-Scholes example carries over to large, production-scale calculation trees, with compilation time staying acceptable; if real tapes are far larger or compile much slower, the practical claim collapses.

Editorial extensions

If this is right

Practitioners can add Vanna, Volga, and other mixed second-order Greeks to a Black-Scholes valuation at about 20% extra runtime, without writing derivative formulas by hand.
Overnight computation of Taylor expansions for portfolios becomes practical: a four-input expansion to order 5 costs about 17 times the primal evaluation, enabling fast intraday polynomial repricing.
Institutions that avoid source-code generation for security or compliance reasons can still use reverse-mode AD with near-handwritten performance.
The single-pass Taylor backpropagation keeps runtime polynomial in derivative order, unlike backward-over-forward approaches that grow exponentially with order, so higher orders remain tractable.
AD-HOC can be embedded as an external function inside dynamic AD tools, supplying fast and even high-order derivatives for those tapes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The compile-time approach invites a direct stress test: scaling the benchmark to larger option portfolios would reveal whether compilation time, flagged in the paper as the 'main difficulty' for higher orders, remains acceptable in production.
The trick of publishing a symbolically simplified Vega as a second output type is a general recipe: any closed-form simplification of a derivative can be fed back into AD-HOC as an extra type, potentially extending the speedup beyond Black-Scholes.
Because the BackPropagator is decoupled from the CalcTree and reusable across multiple trees, checkpointed reverse-mode for time-stepping PDEs or Monte Carlo paths is a natural high-order extension, though the paper does not develop it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The thing to know about arXiv:2412.05300 is that it is a legitimate software contribution: AD-HOC implements high-order Taylor backpropagation with a compile-time 'zero-data' expression template design where every input gets a unique type. That combination appears to be new, and the paper explains the design clearly enough that a competent C++ programmer could reproduce it. The code is public, the algorithm description is correct, and the author is honest about prior art (Arbogast, COSY INFINITY, expression templates in other AD tools). Those are real strengths.

The weak spot is the evidence for the central performance claim. The Section 5 table is the whole case, but it is single-run timings with no error bars, and the prose contradicts the table. The text says RR(order) stays between 2.0x and 2.5x and decreases for AD-HOC from order 3 onward; the table shows RR(3)=1.56, RR(4)=2.67, RR(5)=2.39. That is neither in the claimed band nor decreasing. Since the headline '16.66x vs 634.1x at order 5' is just a ratio of two point estimates, the claimed scaling advantage over ADOL-C is not actually established. The paper also concedes that compile times are the current barrier to higher orders, so the 'arbitrary order' part of the abstract is aspirational, not demonstrated beyond order 5.

The Black-Scholes comparison in Section 4 is fair in one sense and overdrawn in another. AD-HOCv1 is about 20% slower than the specialized handwritten version; AD-HOCv2 only matches by embedding a symbolically simplified Vega expression into the type. That is a legitimate use of the tool, but it does not support the unqualified claim that AD-HOC runs 'with similar speeds as handwritten code' in general.

None of this is fatal. The design is interesting, the code exists, and the benchmark methodology is at least described. But the paper currently overstates what is proven. The fix is straightforward: repeated runs with variance, a small suite of benchmark problems beyond Black-Scholes, a corrected RR table, and a clear statement about which orders are actually reachable today. With that, this could be a solid contribution to the AD community.

Who is this for? Researchers and practitioners doing high-order derivatives in C++, mainly computational finance. It deserves a serious referee, but the referee should be told to focus on the benchmark statistics.

Referee Report

4 major / 5 minor

Summary. The paper presents AD-HOC, a header-only C++17/20 expression-template library for automatic differentiation. The core idea is to encode the entire calculation as a compile-time type, construct a tape at compile time, and perform high-order Taylor backpropagation in a single reverse pass. The author claims arbitrary-order derivatives, performance comparable to handwritten code, and competitive runtimes against established AD tools (ADOL-C, dco/c++, CoDiPack, Tapenade, Enzyme). The paper includes a pedagogical derivation of second-order backpropagation, interface examples using Black-Scholes/Vanna-Volga, and a runtime comparison for full tensors up to order 5.

Significance. If the performance claims hold, the tool could be practically useful in finance for computing greeks and Taylor expansions without source-code generation, and the compile-time type approach is a distinctive design. The paper provides open-source code and an honest discussion of limitations. The central algorithm (Taylor backpropagation) is standard but correctly explained. The main significance is engineering rather than new mathematics.

major comments (4)

[Section 5, Table 1] The claim in the text that 'RR(order) values remain between 2.0x and 2.5x and decrease for both AD-HOC and ADOL-C when calculating orders 3 and beyond' is not supported by the table's own numbers. For AD-HOC, RR(3)=1.56, RR(4)=2.67, and RR(5)=2.39, which falls outside the stated band and is not monotonically decreasing. This inconsistency affects the interpretation of the scaling discussion and should be corrected (either the prose, the table, or both), with a careful statement of which tool actually shows the decreasing trend.
[Section 5, runtime comparison table] The R(N) and RR(N) ratios are derived from timings described only as 'multiple repetitions with randomized inputs', with no number of repetitions, standard deviations, or confidence intervals. Since the central performance claims rest on these point estimates (e.g., AD-HOC's R(5)=16.66x versus ADOL-C's 634.1x), the measurements need repeated-trial statistics and a description of the variance across runs before the speed advantage can be considered established.
[Section 4, Black-Scholes benchmark] The abstract's claim that AD-HOC 'can run with similar speeds as handwritten code' is only supported in the AD-HOCv2 configuration, where the user manually encodes a symbolically simplified Vega expression as a type. In the straightforward AD-HOCv1 configuration, the timings show AD-HOCv1 is slower than HANDv2 at 10M and 100M repetitions (431 vs 358 µs and 4224 vs 3600 µs, respectively). The paper should qualify the 'similar speeds' claim so that it does not imply a fully automatic general tool achieves this on arbitrary user code.
[Section 5 and Section 6, arbitrary-order claim] The highest order demonstrated in the benchmark is 5, and the text states that 'the main difficulty being compilation times' prevents reaching higher orders. The abstract's 'arbitrary order' feature is therefore not established by the presented evidence. Please either provide data on how the tool scales to higher orders (e.g., order 6-8 with compilation times) or soften the claim to 'orders practically limited by compile time'.

minor comments (5)

[Section 2, paragraph 1] The phrase 'Louis Fran¸ co is Antoine Arbogast' contains a typo (should be 'François').
[Section 4, timing table] The sentence 'This version only increases the runtime by close to 45%' is inconsistent with the 1M and 10M rows where the increases are about 22% and 40%; the claim should be tied to a specific repetition count or expressed as a range.
[Section 3.2] The displayed type `C< double, Arg< double>{5.0e-1}>` would be clearer if the template parameter notation (what `C`, `double`, and `Arg` stand for) were explained once before use.
[Section 3.4] The use of `d(x_1)*d(x_2)` to denote a mixed partial derivative is initially confusing because `*` normally denotes multiplication; a brief note that the differential operator is applied as a product would help.
[Section 5, Table 1] The header `R (order)` is redundant with the row label `order`; consider simplifying the table header and explicitly defining `R` and `RR` in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AD-HOC's core derivation is self-contained and its performance claims rest on external benchmark measurements rather than fitted parameters or self-cited theorems.

full rationale

The paper's central algorithm is a Taylor-expansion backpropagation whose derivation is spelled out in Section 2 with explicit series substitutions (equations 2.1-2.4). This derivation does not assume the conclusion; it constructs the derivative propagation from the chain rule and Taylor coefficients. The high-order technique is credited to Arbogast, COSY INFINITY, and ADOL-C, and no load-bearing result is imported from a self-citation. The benchmark evidence in Sections 4 and 5 compares AD-HOC against external AD packages (ADOL-C, dco/c++, CoDiPack, Tapenade, Enzyme), and the code is publicly available; these are independent, falsifiable measurements, not predictions derived from fitted inputs. The AD-HOCv2 comparison with HANDv2 uses a manually simplified Vega expression encoded as a type, but the paper explicitly discloses this and explains that the symbolic simplification gives handwritten code an 'unfair' advantage; matching that specialized version does not reduce the general claim to a circular construction. No parameter is fitted and then renamed as a prediction. The only weaknesses found are evidentiary, not circular: Section 5's prose states RR(order) remains between 2.0x and 2.5x and decreases for orders 3 and beyond, while the table reports RR(3)=1.56, RR(4)=2.67, and RR(5)=2.39, and the paper concedes that higher orders are blocked mainly by compilation times. These concerns affect the strength of the performance claim, but they do not constitute circular reasoning because the measurements are externally grounded and the limitations are stated rather than hidden.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper does not introduce new physical or mathematical entities. It relies on standard calculus (Taylor expansion) and standard compiler behavior assumptions. No free parameters are fitted to data.

assumptions (3)

standard math Taylor expansion of elementary functions with truncation at the required order is valid.
The backpropagation algorithm in Section 2 relies on this standard calculus result to compute high-order derivatives.
domain assumption The C++ compiler performs expected optimizations (inlining, constant folding) on expression templates.
The performance claims assume the compiler can statically build an efficient computation tree; this is a load-bearing assumption for the runtime comparison.
domain assumption The benchmark environment (Apple M3 Pro, GCC 13.2, -O3) is representative of typical production environments for AD users.
The runtime ratios are measured on a single platform and compiler; the paper does not assess variability across environments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AD-HOC: A C++ Expression Template package for high-order derivatives backpropagation." pith.science (2026). https://pith.science/paper/MUN3WQ6V

@misc{pith2026241205300,
  author       = {Pith},
  title        = {Pith review of: AD-HOC: A C++ Expression Template package for high-order derivatives backpropagation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MUN3WQ6V}},
  note         = {Machine review of arXiv:2412.05300}
}

read the original abstract

This document presents a new C++ Automatic Differentiation (AD) tool, AD-HOC (Automatic Differentiation for High-Order Calculations). This tool aims to have the following features: -Calculation of user specified derivatives of arbitrary order -To be able to run with similar speeds as handwritten code -All derivatives calculations are computed in a single backpropagation tree pass -No source code generation is used, relying heavily on the C++ compiler to statically build the computation tree before runtime -A simple interface -The ability to be used \textit{in conjunction} with other established, general-purpose dynamic AD tools -Header-only library, with no external dependencies -Open source, with a business-friendly license

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

[1]

Charpentier, J.P

I. Charpentier, J.P. Friedelmeyer, J. Gustedt, Arbogast – Origine d’un outil de d´ erivation automatique , Inria (2016)

work page 2016
[2]

M. Berz, K. Makino, K. Shamseddine, G. Hoﬀstatter, W. Wan , COSY INFINITY and Its Applications in Nonlinear Dynamics, SIAM (1996)

work page 1996
[3]

Sagebaum, T

M. Sagebaum, T. Albring, N. R. Gauger, Expression templates for primal value taping in the reverse mode of algorithmic diﬀerentiation , Advances in Algorithmic Diﬀerentiation Optimization Met hods & Software (2018), pp. 1207-1231

work page 2018
[4]

Phipps, R

E. Phipps, R. Pawlowski, Eﬃcient Expression Templates for Operator Overloading-ba sed Automatic Diﬀerentiation , Recent Advances in Algorithmic Diﬀerentiation (2012), pp. 309-319

work page 2012
[5]

Hogan, Fast Reverse-Mode Automatic Diﬀerentiation using Expressi on Templates in C++ , ACM Transactions on Mathematical Software, Vol

R. Hogan, Fast Reverse-Mode Automatic Diﬀerentiation using Expressi on Templates in C++ , ACM Transactions on Mathematical Software, Vol. 40, No. 4, Article 26 (2014)

work page 2014
[6]

Aubert, N

P. Aubert, N. Di C´ esar´ e,Expression Templates and Forward Mode Automatic Diﬀerentia tion, Automatic Diﬀerenti- ation of Algorithms: From Simulation to Optimization (2002 ), pp. 311-315

work page 2002
[7]

Savine, Modern Computational Finance: AAD and Parallel Simulation s, Wiley (2018)

A. Savine, Modern Computational Finance: AAD and Parallel Simulation s, Wiley (2018)

work page 2018
[8]

L. F. A. Arbogast, Du calcul des d´ erivations, Imprimerie de Levrault fr` eres, Strasbourg, An VIII (1800 )

work page

Show all 15 references

[9]

Stone, https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1045r1.html, P1045R0 EWG (2018)

D. Stone, https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1045r1.html, P1045R0 EWG (2018)

2018
[10]

Black, M

F. Black, M. Scholes, The Pricing of Options and Corporate Liabilities , Journal of Political Economy Vol. 81, No. 3 (May - Jun., 1973), pp. 637-654

1973
[11]

Castagna, F

A. Castagna, F. Mercurio, The Vanna-Volga method for implied volatilities: tractabi lity and robustness , ResearchGate (2007)

2007
[12]

M. Berz, Forward algorithms for high orders and many variables with a pplication to beam physics , Automatic Diﬀerentiation of Algorithms: Theory, Implementation, an d Applications (1991), pp. 147-156

1991
[13]

Faure, U

C. Faure, U. Naumann, Minimizing the Tape Size , Automatic Diﬀerentiation of Algorithms: From Simulation to Optimization (2002), pp. 293-298

2002
[14]

Naumann, Call Tree Reversal is NP-Complete , Advances in Automatic Diﬀerentiation (2008), pp

U. Naumann, Call Tree Reversal is NP-Complete , Advances in Automatic Diﬀerentiation (2008), pp. 13-22

2008
[15]

Griewank, A

A. Griewank, A. Walther, Evaluating Derivatives: Principles and Techniques of Algo rithmic Diﬀerentiation , SIAM (2008) Copyright © 2025 by SIAM Unauthorized reproduction of this article is prohibited

2008

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Charpentier, J.P

I. Charpentier, J.P. Friedelmeyer, J. Gustedt, Arbogast – Origine d’un outil de d´ erivation automatique , Inria (2016)

work page 2016

[2] [2]

M. Berz, K. Makino, K. Shamseddine, G. Hoﬀstatter, W. Wan , COSY INFINITY and Its Applications in Nonlinear Dynamics, SIAM (1996)

work page 1996

[3] [3]

Sagebaum, T

M. Sagebaum, T. Albring, N. R. Gauger, Expression templates for primal value taping in the reverse mode of algorithmic diﬀerentiation , Advances in Algorithmic Diﬀerentiation Optimization Met hods & Software (2018), pp. 1207-1231

work page 2018

[4] [4]

Phipps, R

E. Phipps, R. Pawlowski, Eﬃcient Expression Templates for Operator Overloading-ba sed Automatic Diﬀerentiation , Recent Advances in Algorithmic Diﬀerentiation (2012), pp. 309-319

work page 2012

[5] [5]

Hogan, Fast Reverse-Mode Automatic Diﬀerentiation using Expressi on Templates in C++ , ACM Transactions on Mathematical Software, Vol

R. Hogan, Fast Reverse-Mode Automatic Diﬀerentiation using Expressi on Templates in C++ , ACM Transactions on Mathematical Software, Vol. 40, No. 4, Article 26 (2014)

work page 2014

[6] [6]

Aubert, N

P. Aubert, N. Di C´ esar´ e,Expression Templates and Forward Mode Automatic Diﬀerentia tion, Automatic Diﬀerenti- ation of Algorithms: From Simulation to Optimization (2002 ), pp. 311-315

work page 2002

[7] [7]

Savine, Modern Computational Finance: AAD and Parallel Simulation s, Wiley (2018)

A. Savine, Modern Computational Finance: AAD and Parallel Simulation s, Wiley (2018)

work page 2018

[8] [8]

L. F. A. Arbogast, Du calcul des d´ erivations, Imprimerie de Levrault fr` eres, Strasbourg, An VIII (1800 )

work page