{"id":"0ac4d314-5ddb-472e-844a-4f3958fda031","arxiv_id":"2412.05300","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"AD-HOC uses compile-time expression templates and Taylor expansions to compute arbitrary-order derivatives in a single backpropagation pass, with benchmarks showing near-handwritten runtime.","lead":"AD-HOC is a new header-only C++ library that computes high-order derivatives of arbitrary mathematical expressions using a single backward pass and no source code generation. The library promises runtimes close to handwritten code, which could matter for high-frequency finance and other performance-sensitive applications.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5's headline performance ratio rests on single-run timings with no error bars, and the reported RR scaling is internally inconsistent; the paper also concedes compilation time blocks higher orders, so neither the speed nor the arbitrary-order claim is yet established.","rationale":"The reader's weakest assumption already flagged both the lack of error bars on the Black-Scholes benchmark and the compile-time scaling limit for higher orders. My stress-test confirms that concern and adds a concrete internal inconsistency: the Section 5 prose says AD-HOC's RR(order) stays between 2.0x and 2.5x and decreases for orders 3 and beyond, but the table shows RR(3)=1.56, RR(4)=2.67, RR(5)=2.39. That inconsistency weakens the evidential value of the very table used to support the central runtime claim. The public code and open license are genuine independent assets, and a proper rerun with confidence intervals could restore confidence, so I do not recommend moving the verdict to REJECT or UNVERDICTED. The appropriate status remains CONDITIONAL: the paper is a credible software contribution, but the headline performance comparison and the arbitrary-order claim need stronger empirical support before acceptance.","tokens_in":41,"tokens_out":6945,"duration_ms":133467,"concrete_test":"Run the public case_studies/2024ADChicago tensor benchmark at least 10 times per configuration (BASE, AD-HOC orders 1-5, ADOL-C orders 1-5), reporting mean and standard deviation for every R(N) and RR(N). Then compile AD-HOC variants for orders 6 and 7 with the same GCC -O3 flags, recording wall-clock compile time and peak memory. If the confidence intervals for AD-HOC's R(5) are wide enough to overlap the trend implied by RR(4) and RR(5), or if compile time grows worse than roughly O(order^2) and becomes impractical by order 7, the central performance and scalability claims should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing evidence for the central performance claim is Section 5's table of R(N) ratios, but that table is not supported by repeated-run statistics and is internally inconsistent with its own prose. For AD-HOC, the prose claims 'RR(order) values remain between 2.0x and 2.5x and decrease for both AD-HOC and ADOL-C when calculating orders 3 and beyond', yet the table gives RR(3)=1.56, RR(4)=2.67, and RR(5)=2.39: not in the claimed band and not monotonically decreasing. This makes the headline 'AD-HOC 16.66x vs ADOL-C 634.1x at order 5' a comparison of noisy point estimates rather than an established scaling law. The paper also states in Section 5 that 'the main difficulty being compilation times' prevents reaching higher orders, so the 'arbitrary order' part of the central claim is explicitly not demonstrated beyond order 5. Additionally, the Black-Scholes AD-HOCv2 comparison uses a hand-simplified Vega expression encoded as a type; without that manual simplification, AD-HOCv1 is slower than HANDv2 at large repetition counts, so the 'similar speeds as handwritten code' claim is not established for the general tool. These are not fatal flaws, but the current evidence does not support unconditional acceptance of the performance claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents AD-HOC, a header-only C++17/20 expression-template library for automatic differentiation. The core idea is to encode the entire calculation as a compile-time type, construct a tape at compile time, and perform high-order Taylor backpropagation in a single reverse pass. The author claims arbitrary-order derivatives, performance comparable to handwritten code, and competitive runtimes against established AD tools (ADOL-C, dco/c++, CoDiPack, Tapenade, Enzyme). The paper includes a pedagogical derivation of second-order backpropagation, interface examples using Black-Scholes/Vanna-Volga, and a runtime comparison for full tensors up to order 5.","tokens_in":12198,"tokens_out":5448,"duration_ms":44121,"significance":"If the performance claims hold, the tool could be practically useful in finance for computing greeks and Taylor expansions without source-code generation, and the compile-time type approach is a distinctive design. The paper provides open-source code and an honest discussion of limitations. The central algorithm (Taylor backpropagation) is standard but correctly explained. The main significance is engineering rather than new mathematics.","major_comments":[{"comment":"The claim in the text that 'RR(order) values remain between 2.0x and 2.5x and decrease for both AD-HOC and ADOL-C when calculating orders 3 and beyond' is not supported by the table's own numbers. For AD-HOC, RR(3)=1.56, RR(4)=2.67, and RR(5)=2.39, which falls outside the stated band and is not monotonically decreasing. This inconsistency affects the interpretation of the scaling discussion and should be corrected (either the prose, the table, or both), with a careful statement of which tool actually shows the decreasing trend.","section":"Section 5, Table 1"},{"comment":"The R(N) and RR(N) ratios are derived from timings described only as 'multiple repetitions with randomized inputs', with no number of repetitions, standard deviations, or confidence intervals. Since the central performance claims rest on these point estimates (e.g., AD-HOC's R(5)=16.66x versus ADOL-C's 634.1x), the measurements need repeated-trial statistics and a description of the variance across runs before the speed advantage can be considered established.","section":"Section 5, runtime comparison table"},{"comment":"The abstract's claim that AD-HOC 'can run with similar speeds as handwritten code' is only supported in the AD-HOCv2 configuration, where the user manually encodes a symbolically simplified Vega expression as a type. In the straightforward AD-HOCv1 configuration, the timings show AD-HOCv1 is slower than HANDv2 at 10M and 100M repetitions (431 vs 358 µs and 4224 vs 3600 µs, respectively). The paper should qualify the 'similar speeds' claim so that it does not imply a fully automatic general tool achieves this on arbitrary user code.","section":"Section 4, Black-Scholes benchmark"},{"comment":"The highest order demonstrated in the benchmark is 5, and the text states that 'the main difficulty being compilation times' prevents reaching higher orders. The abstract's 'arbitrary order' feature is therefore not established by the presented evidence. Please either provide data on how the tool scales to higher orders (e.g., order 6-8 with compilation times) or soften the claim to 'orders practically limited by compile time'.","section":"Section 5 and Section 6, arbitrary-order claim"}],"minor_comments":[{"comment":"The phrase 'Louis Fran¸ co is Antoine Arbogast' contains a typo (should be 'François').","section":"Section 2, paragraph 1"},{"comment":"The sentence 'This version only increases the runtime by close to 45%' is inconsistent with the 1M and 10M rows where the increases are about 22% and 40%; the claim should be tied to a specific repetition count or expressed as a range.","section":"Section 4, timing table"},{"comment":"The displayed type `C< double, Arg< double>{5.0e-1}>` would be clearer if the template parameter notation (what `C`, `double`, and `Arg` stand for) were explained once before use.","section":"Section 3.2"},{"comment":"The use of `d(x_1)*d(x_2)` to denote a mixed partial derivative is initially confusing because `*` normally denotes multiplication; a brief note that the differential operator is applied as a product would help.","section":"Section 3.4"},{"comment":"The header `R (order)` is redundant with the row label `order`; consider simplifying the table header and explicitly defining `R` and `RR` in the caption.","section":"Section 5, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a software/benchmark contribution. The novelty over existing expression-template AD tools (e.g., Sagebaum et al., Hogan) is incremental but the high-order single-pass combination is not widely available. The benchmark section needs statistical rigor before acceptance. The informal tone and several formatting glitches suggest light editing. The author is the sole developer of the tool, which is normal for such papers but means independent validation would strengthen confidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about arXiv:2412.05300 is that it is a legitimate software contribution: AD-HOC implements high-order Taylor backpropagation with a compile-time 'zero-data' expression template design where every input gets a unique type. That combination appears to be new, and the paper explains the design clearly enough that a competent C++ programmer could reproduce it. The code is public, the algorithm description is correct, and the author is honest about prior art (Arbogast, COSY INFINITY, expression templates in other AD tools). Those are real strengths.\n\nThe weak spot is the evidence for the central performance claim. The Section 5 table is the whole case, but it is single-run timings with no error bars, and the prose contradicts the table. The text says RR(order) stays between 2.0x and 2.5x and decreases for AD-HOC from order 3 onward; the table shows RR(3)=1.56, RR(4)=2.67, RR(5)=2.39. That is neither in the claimed band nor decreasing. Since the headline '16.66x vs 634.1x at order 5' is just a ratio of two point estimates, the claimed scaling advantage over ADOL-C is not actually established. The paper also concedes that compile times are the current barrier to higher orders, so the 'arbitrary order' part of the abstract is aspirational, not demonstrated beyond order 5.\n\nThe Black-Scholes comparison in Section 4 is fair in one sense and overdrawn in another. AD-HOCv1 is about 20% slower than the specialized handwritten version; AD-HOCv2 only matches by embedding a symbolically simplified Vega expression into the type. That is a legitimate use of the tool, but it does not support the unqualified claim that AD-HOC runs 'with similar speeds as handwritten code' in general.\n\nNone of this is fatal. The design is interesting, the code exists, and the benchmark methodology is at least described. But the paper currently overstates what is proven. The fix is straightforward: repeated runs with variance, a small suite of benchmark problems beyond Black-Scholes, a corrected RR table, and a clear statement about which orders are actually reachable today. With that, this could be a solid contribution to the AD community.\n\nWho is this for? Researchers and practitioners doing high-order derivatives in C++, mainly computational finance. It deserves a serious referee, but the referee should be told to focus on the benchmark statistics.","headline":"A real and clearly described C++ AD library with a novel compile-time design, but the headline performance ratio rests on single-run timings and a table that contradicts its own prose.","tokens_in":29,"tokens_out":3155,"would_cite":false,"duration_ms":56877,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AD-HOC is a C++ automatic differentiation library that computes user-specified high-order derivatives in a single backpropagation pass, with runtimes close to handwritten code.","keywords":["automatic differentiation","high-order derivatives","expression templates","Taylor backpropagation","compile-time computation tree","Black-Scholes","Vanna-Volga","C++ template metaprogramming"],"falsifier":"Run AD-HOC on a large calculation tape, such as a portfolio of hundreds of Black-Scholes options, request order-5 tensors, and measure both compile time and the runtime ratio R(5). A compile time that grows superlinearly with the number of nodes, or an R(5) far above the reported 16.66x, would falsify the claim that AD-HOC keeps handwritten-code-level performance at scale.","tokens_in":11729,"feed_emoji":"🧮","tokens_out":10532,"duration_ms":238445,"temperature":0.7,"pith_summary":"AD-HOC is a C++ automatic differentiation library that computes user-selected derivatives of any order in a single reverse pass. Its central claim is that by encoding the entire calculation as a compile-time type, with each input distinguished by a unique type and constants stored as template parameters, it can match the runtime of handwritten derivative code. The paper demonstrates this on the Black-Scholes formula, where computing price, Vega, Vanna, and Volga with AD-HOCv2 takes 44 microseconds per million evaluations versus 46 for purpose-built handwritten code. For a general four-input tensor up to order 5, AD-HOC's runtime ratio is 16.66x, compared to 634x for the dynamic ADOL-C tool. A sympathetic reader would take this as evidence that static, type-based AD can deliver both high-order derivatives and practical speed without source-code generation.","feed_headline":"High-order derivatives in one pass, at handwritten-code speed","feed_subtitle":"AD-HOC encodes calculations as compile-time types, so order-5 tensors cost 16.7x the primal run. ADOL-C takes 634x.","key_machinery":"The central object is the compile-time calculation tree built from expression templates: a C++ type that contains zero data yet fully describes the computation, including constants encoded as template parameter values. The pruning mechanism is the BackPropagator, which takes only the requested derivative seeds and runs truncated Taylor expansions through each node, truncating monomials that do not contribute to the requested derivatives. Memory is handled by a CalcTree that allocates a single stack std::array sized at compile time, plus a buffer sized to the maximum backpropagation slice. Because the tree is fully static, the compiler can omit storing intermediate values that backpropagation does not need, which the paper credits for the runtime advantage.","core_discovery":"The paper's central claim is that high-order derivatives can be computed efficiently by representing the full calculation tree as a data-less C++ type and then performing a Taylor-expansion backpropagation over that static tree. In AD-HOC, each input variable has a unique type, constants are encoded as template parameters, and expressions build nested types such as mul_t<double_t<\"S\">, erfc_t<...>> that describe the whole computation with zero runtime data. The backpropagator receives only the derivative seeds the user needs, such as d(V)*d(S) or d<2>(V), propagates truncated Taylor series through each node by substituting one series into another, and multiplies the resulting coefficients by factorials to obtain the requested derivatives in a single tree pass. The paper reports that this yields Black-Scholes price plus three Greek sensitivities at a runtime 1.19x to 1.45x the plain price evaluation, essentially matching handwritten code, and that a four-input full tensor to order 5 costs 16.66x the primal evaluation versus 634x for ADOL-C.","pith_inferences":["The compile-time approach invites a direct stress test: scaling the benchmark to larger option portfolios would reveal whether compilation time, flagged in the paper as the 'main difficulty' for higher orders, remains acceptable in production.","The trick of publishing a symbolically simplified Vega as a second output type is a general recipe: any closed-form simplification of a derivative can be fed back into AD-HOC as an extra type, potentially extending the speedup beyond Black-Scholes.","Because the BackPropagator is decoupled from the CalcTree and reusable across multiple trees, checkpointed reverse-mode for time-stepping PDEs or Monte Carlo paths is a natural high-order extension, though the paper does not develop it."],"forward_implications":["Practitioners can add Vanna, Volga, and other mixed second-order Greeks to a Black-Scholes valuation at about 20% extra runtime, without writing derivative formulas by hand.","Overnight computation of Taylor expansions for portfolios becomes practical: a four-input expansion to order 5 costs about 17 times the primal evaluation, enabling fast intraday polynomial repricing.","Institutions that avoid source-code generation for security or compliance reasons can still use reverse-mode AD with near-handwritten performance.","The single-pass Taylor backpropagation keeps runtime polynomial in derivative order, unlike backward-over-forward approaches that grow exponentially with order, so higher orders remain tractable.","AD-HOC can be embedded as an external function inside dynamic AD tools, supplying fast and even high-order derivatives for those tapes."],"supporting_citations":[{"why":"Supplies the historical series-substitution method that the backpropagation algorithm implements.","marker":"[1]"},{"why":"Shows prior high-order Taylor backpropagation in COSY INFINITY, the approach AD-HOC combines with expression templates.","marker":"[2]"},{"why":"Shows expression templates in fast reverse-mode AD, the compile-time technique AD-HOC reworks with data-less types.","marker":"[5]"},{"why":"Provides the Black-Scholes formula used as the main benchmark and application example.","marker":"[10]"},{"why":"Defines the Vanna-Volga method whose target derivatives, Vega, Vanna and Volga, structure the benchmark.","marker":"[11]"},{"why":"Supplies the theoretical polynomial runtime scaling with derivative order that the paper cites.","marker":"[12]"},{"why":"Provides the algorithmic differentiation theory predicting quadratic runtime scaling in order for Taylor backpropagation.","marker":"[15]"}],"fun_headline_variants":["One pass, compile-time types: order-5 at 16.66x vs 634x","Greeks near price time: 1.19-1.45x via C++ type backprop","No codegen, no runtime tree: AD-HOC compiles derivatives","High-order derivatives at handwritten speed: one pass, zero codegen"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the paper's measured performance on a small Black-Scholes example carries over to large, production-scale calculation trees, with compilation time staying acceptable; if real tapes are far larger or compile much slower, the practical claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["One pass, compile-time types: order-5 at 16.66x vs 634x","Greeks near price time: 1.19-1.45x via C++ type backprop","No codegen, no runtime tree: AD-HOC compiles derivatives","High-order derivatives at handwritten speed: one pass, zero codegen"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000471,"raw_usage":{"total_tokens":2316,"prompt_tokens":890,"completion_tokens":1426,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":1335}},"tokens_in":506,"tokens_out":1426,"duration_ms":27562,"temperature":1.0,"reasoning_tokens":1335,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:18:08.414329+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run AD-HOC on a large calculation tape, such as a portfolio of hundreds of Black-Scholes options, request order-5 tensors, and measure both compile time and the runtime ratio R(5). A compile time that grows superlinearly with the number of nodes, or an R(5) far above the reported 16.66x, would falsify the claim that AD-HOC keeps handwritten-code-level performance at scale.","supporting_citations":[{"cited_title":"Charpentier, J.P","cited_arxiv_id":null,"evidence_quote":"Supplies the historical series-substitution method that the backpropagation algorithm implements."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows prior high-order Taylor backpropagation in COSY INFINITY, the approach AD-HOC combines with expression templates."},{"cited_title":"Hogan, Fast Reverse-Mode Automatic Differentiation using Expressi on Templates in C++ , ACM Transactions on Mathematical Software, Vol","cited_arxiv_id":null,"evidence_quote":"Shows expression templates in fast reverse-mode AD, the compile-time technique AD-HOC reworks with data-less types."},{"cited_title":"Black, M","cited_arxiv_id":null,"evidence_quote":"Provides the Black-Scholes formula used as the main benchmark and application example."},{"cited_title":"Castagna, F","cited_arxiv_id":null,"evidence_quote":"Defines the Vanna-Volga method whose target derivatives, Vega, Vanna and Volga, structure the benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the theoretical polynomial runtime scaling with derivative order that the paper cites."},{"cited_title":"Griewank, A","cited_arxiv_id":null,"evidence_quote":"Provides the algorithmic differentiation theory predicting quadratic runtime scaling in order for Taylor backpropagation."}],"review_version":1}