{"id":"08a13428-72bd-4b9d-b3f9-124a14d433e5","arxiv_id":"2506.18522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DDOT, a dual-decoder transformer with derivative prediction, reports improved symbolic ODE reconstruction on ODEBench and introduces a divergence-based metric, DIV-diff, for evaluating ODE hypotheses.","lead":"DDOT is a transformer model that reads time-series trajectories and writes down the symbolic ordinary differential equations that may have produced them, assisted by a second decoder that predicts derivative values. The paper also introduces DIV-diff, a metric that compares the divergence of two candidate ODEs across a region instead of along a single trajectory.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DIV-diff conflates divergence with vector-field similarity; the 3.55% DIV-diff gain may reflect metric weakness rather than better ODE recovery, and the derivative-decoder's claimed link to DIV-diff is not derived.","rationale":"The reader's weakest_assumption identifies exactly the concern I consider load-bearing: DIV-diff, as a divergence-based metric, cannot distinguish vector fields that share divergence but differ in rotational or shear components. My analysis adds specificity: the paper supplies no formula for DIV-diff, no grid or estimator details, no derivation connecting the derivative cross-entropy loss to divergence accuracy, and no empirical check that DIV-diff tracks trajectory-level recovery. The reconstruction and generalization improvements are plausible incremental gains from an auxiliary decoder and are secondary. The central quantitative evidence for the new metric and the headline 3.55% reduction is therefore not yet interpretable. This does not warrant rejection, because the underlying DDOT architecture is a reasonable extension of ODEFormer and the Tables 1 results, if reproducible, support a modest improvement. But the paper needs the precise metric definition, a control experiment with divergence-preserving perturbations, and a correlation analysis against rollout metrics before the DIV-diff claims can be accepted as evidence of better ODE recovery. Hence the verdict stays CONDITIONAL with the conditions tightened to include metric specification and validation. I agree with the reader's identification; I did not find an independent stronger weakness in the model's core architecture beyond the metric issue, and I do not see internal inconsistency in the training loss itself, only an unvalidated claim about what the derivative decoder achieves.","tokens_in":8241,"tokens_out":2157,"duration_ms":23166,"concrete_test":"Recompute Table 2 with the precise DIV-diff definition (grid spacing, domain bounds, divergence estimator, RLMSE base) and add a control experiment on a divergence-equivalence class: for each ODEBench system, construct a perturbed ODE obtained by adding a divergence-free (e.g., Hamiltonian-type or shear) component that leaves the divergence field unchanged. If the perturbed ODEs achieve DIV-diff near zero while their rollout R^2 and generalization accuracy degrade sharply, then DIV-diff is not a sufficient measure of ODE recovery and the claimed 3.55% DIV-diff advantage loses its evidential weight. Additionally, report per-system correlation between DIV-diff and rollout P(R^2>0.9) across all 63 ODEBench systems; low or negative correlation would show the two metrics are not measuring the same construct.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central quantitative claims for DDOT rest on two legs: P(R^2>0.9) improvements (Tables 1) and DIV-diff improvements (Table 2). The reconstruction/generalization gains (4.58% and 1.62%) are modest and consistent with an incremental auxiliary-task extension, so they are not the main risk. The load-bearing concern is DIV-diff itself. Divergence is the trace of the Jacobian; it discards all off-diagonal, rotational, and shear structure of the vector field. Two ODEs can have identical divergence at every point yet completely different trajectories (e.g., the rotation field (-y, x) and the scaled field (-2y, 2x) have divergences 0 and 0, but very different dynamics; more generally any divergence-free field can be added to a gradient field without changing divergence). The paper asserts in Sections 1 and 5.3 that DIV-diff is 'more comprehensive' and 'reliable,' but provides no argument that divergence difference is a sufficient statistic for ODE recovery. Moreover, no precise formula for DIV-diff is given: the metric is only described as 'RLMSE of the divergence difference,' leaving the grid, the divergence estimator (autodiff or finite difference), and the averaging domain unspecified. Without the precise definition, Tables 2 and the claimed 3.55% absolute reduction cannot be reproduced or interpreted. Even granting the numbers, a smaller DIV-diff could mean the predicted ODE is closer in divergence while being worse in rotational components; the paper never checks whether DIV-diff correlates with rollout-based accuracy or with the symbolic form error. A second concern is internal: the derivative-decoder is trained with cross-entropy on discretized derivative tokens (Section 4.2), yet the paper claims this 'approximates the true ODE vector field' and aligns divergence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DDOT, a transformer-based model for symbolic regression of ordinary differential equations, which augments ODEFormer-style symbolic decoding with a second decoder that predicts the derivative sequence of the system. The authors also introduce DIV-diff, a proposed evaluation metric that compares the divergence (trace of the Jacobian) of the predicted and ground-truth vector fields over a grid of points, arguing that this is more comprehensive than single-trajectory rollout. Experiments on ODEBench report that DDOT outperforms ODEFormer, SINDy, and ProGED in reconstruction and generalization P(R^2>0.9), and achieves lower DIV-diff values, with a real-world anesthesia case study presented as an additional demonstration.","tokens_in":8572,"tokens_out":6165,"duration_ms":63263,"significance":"If the results hold, the dual-decoder architecture is a reasonable incremental extension of ODEFormer, and the proposed evaluation direction addresses a genuine limitation of single-trajectory evaluation. The paper deserves credit for benchmarking on ODEBench against nontrivial baselines and for identifying initial-condition sensitivity as a problem. However, the current significance is limited by three issues: DIV-diff is not given a precise, reproducible definition; the claim that divergence difference is a comprehensive similarity measure is unsupported and in fact false in general; and the reported gains are small and lack any uncertainty quantification. The derivative-decoder loss is not actually a divergence-based loss, so the connection between the auxiliary task and DIV-diff is asserted rather than derived. These are load-bearing concerns for the paper's central quantitative claims.","major_comments":[{"comment":"DIV-diff is never defined precisely enough to be reproduced or interpreted. The text says only that it is the 'Root Logarithm Mean Squared Error (RLMSE) of the divergence difference,' but no formula is given, and the grid region, grid resolution, divergence estimator (autodiff, finite differences, or symbolic differentiation), and averaging domain are all unspecified. Without these details, Table 2 and the claimed 0.0355 reduction in RLMSE cannot be checked, and the metric could be computed in many different ways that would change the rankings.","section":"§5.1 (Evaluation Metrics), Table 2"},{"comment":"The paper claims that DIV-diff is a 'more comprehensive' and 'reliable' evaluation because it is independent of initial conditions, but divergence is only the trace of the Jacobian of the vector field and discards off-diagonal, shear, and rotational components. As a concrete counterexample, the planar fields f(x,y)=(-y,x) and g(x,y)=(x,-y) both have divergence zero at every point, yet one produces rotational dynamics and the other produces saddle dynamics. Therefore two ODEs can have identical divergence everywhere and completely different trajectories, so DIV-diff is not a sufficient similarity measure for ODE recovery. The authors need either to justify why divergence is adequate for their particular systems, or to replace DIV-diff with a full Jacobian or vector-field comparison metric.","section":"§1, §5.3"},{"comment":"The derivative decoder is trained with a standard cross-entropy loss over predicted derivative tokens, and no derivation is provided showing that this loss approximates the divergence of the vector field. Calling Lder a 'divergence difference loss' is misleading: the decoder supervises pointwise derivative vectors, and matching derivative vectors is a stronger condition than matching their divergence, but the paper does not show that the auxiliary task actually drives down DIV-diff. The sentence stating that estimating derivatives 'results in a ground-truth divergence closely aligning with the true divergence' conflates vector-field matching with divergence matching and needs to be supported by analysis or an ablation.","section":"§4.2, Eq. (5)"},{"comment":"The reported improvements are small and no error bars, confidence intervals, or significance tests are provided. The average reconstruction gain is 4.58 percentage points and the average generalization gain is 1.62 percentage points, while the DIV-diff reduction is only 0.0355 in RLMSE units; in Table 2 DDOT is actually worse than ODEFormer at noise=0.01 (2.973 vs 2.703). Without multiple random seeds or statistical tests, the observed differences could easily be within run-to-run variation, especially given the 600,000-step training schedule and the absence of any reported variance.","section":"Tables 1 and 2"},{"comment":"The training data generation process is load-bearing because the model is trained entirely on generated ODEs, yet the paper states 'we omit the detailed generation process.' The distribution over ODE structures, the range of constants, integration parameters, trajectory length, sampling times, and the tokenization of derivative values are not specified. This makes it impossible to assess how the training distribution affects generalization to ODEBench, and it also prevents the experiments from being reproduced by others.","section":"§4.1 (Data Generation)"}],"minor_comments":[{"comment":"The abstract claims an 'absolute reduction of 3.55% in DIV-diff,' but the numbers in Table 2 are RLMSE values around 2.8, and the difference between the ODEFormer and DDOT means is 0.0355, which is about 1.25% relative. The claim should be reworded to state the reduction in RLMSE units or as a relative percentage.","section":"Abstract and §5.2"},{"comment":"In the ODEFormer generalization row, the entries '0.2100.226' for noise=0.01 appear to be missing a separator, which makes the table hard to read.","section":"Table 1"},{"comment":"The color scale and units of the divergence heatmaps are not defined, and the text says only that 'red indicates lower divergence and blue higher'; a colorbar and an explanation of how the divergence was computed for the visualization would be necessary to interpret the figure.","section":"Figure 3"},{"comment":"The anesthesia case study is presented as a demonstration of practical impact, but no dataset source, preprocessing steps, train/test split, or quantitative evaluation are provided, so the interpretability and predictive accuracy claims for this application are not substantiated.","section":"§5.4 (Anesthesia Dataset)"},{"comment":"The phrase 'calculating the divergence... by integrating the ODE over a grid' in §1 is confusing; divergence is a pointwise property of a vector field and does not involve integrating the ODE over a grid. This wording should be corrected to describe evaluating the vector field and its divergence at grid points.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a machine learning journal and presents a plausible incremental improvement over ODEFormer. The main risk is that the proposed DIV-diff metric, which is central to the evaluation and to the claimed link with the derivative decoder, is both underspecified and theoretically insufficient as a measure of ODE similarity. The authors could address this by giving a precise formula, adding a Jacobian-based or full-vector-field comparison, and providing significance testing. If these changes are made, the contribution may become acceptable; as it stands, the quantitative claims are not yet robust enough to support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Name],\n\nQuick take: DDOT is a sensible, incremental extension of ODEFormer—add a second decoder that predicts the derivative sequence, train with cross-entropy on both decoders, and you get a few percent better trajectory reconstruction on ODEBench. The experiments are internally consistent. But the DIV-diff metric, which the paper presents as its other main contribution, is under-defined and theoretically shaky. That keeps the paper at 'conditionally acceptable' rather than 'clearly solid.'\n\nWhat's actually new: the dual-decoder architecture (ODE decoder plus derivative decoder) is a natural auxiliary-task idea, and the paper is the first to name and test a divergence-difference metric for ODE recovery. The authors follow the established ODEFormer pipeline for data generation and tokenization, so the comparison is controlled. The reported gains—4.58% absolute in reconstruction P(R^2>0.9), 1.62% in generalization, 3.55% reduction in DIV-diff—match the tables. I checked the averages; they hold up.\n\nThe weak spots are real but not fatal. The DIV-diff definition is incomplete: no grid bounds or resolution, no specification of how divergence is estimated (autodiff vs finite differences), no exact RLMSE formula. The paper's claim that 'predicting derivatives approximates the true ODE vector field, resulting in ground-truth divergence closely aligning' is asserted, not derived. And the conceptual issue stands: divergence is the trace of the Jacobian, so it ignores rotational and shear components. Two vector fields with identical divergence can have very different dynamics. The metric may still have some diagnostic value, and the paper also reports trajectory-based metrics, so this is a weakness in one component, not a collapse of the whole paper.\n\nAlso: no code yet (promised for camera-ready), no error bars or significance tests, and the anesthesia case study is anecdotal. The gains over ODEFormer are small and might not generalize beyond ODEBench.\n\nWho gets value: researchers working on transformer-based symbolic regression or ODE identification will find this a useful datapoint, especially if they care about auxiliary tasks. The DIV-diff idea, once properly specified and validated, could be useful, but in its current form it is not a reliable evaluation tool.\n\nFor peer review: yes, send it. It's a clear, incremental paper with reproducible-looking tables. A referee should ask for a precise metric definition, code, and a careful discussion of what divergence does and doesn't capture. With those revisions it could be a legitimate workshop- or conference-level contribution.\n\nBest,\n[Your name]","headline":"Small but genuine gains from a derivative-decoding auxiliary task; the DIV-diff metric is under-specified and conceptually weak, though not fatal.","tokens_in":9153,"tokens_out":3507,"would_cite":false,"duration_ms":33751,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding a derivative-prediction decoder to a symbolic ODE transformer yields more accurate equation recovery and a more stable evaluation metric.","keywords":["symbolic regression","ordinary differential equations","transformer","derivative decoder","divergence difference","DIV-diff","dynamic system modeling","ODE discovery"],"falsifier":"Compute DIV-diff between $\\dot{x} = x, \\dot{y} = y$ and $\\dot{x} = x, \\dot{y} = y + x$ over a grid covering a neighbourhood of the origin: both have divergence 2, so DIV-diff would be near zero even though trajectories diverge quadratically in the second component, showing the metric can judge dynamically different ODEs as essentially identical.","tokens_in":8036,"feed_emoji":"🧮","tokens_out":7431,"duration_ms":71279,"temperature":0.7,"pith_summary":"The paper argues that symbolic recovery of ordinary differential equations from trajectory data is best done by a transformer that predicts both the equation and its local derivative field. The proposed DDOT model couples a symbolic ODE decoder with a second decoder that predicts $f(x(t_i))$ at every observed point, training both with cross-entropy so the model is rewarded for matching structure and dynamics at once. On the ODEBench benchmark this yields the reported gains over the prior state of the art: average absolute gains of 4.58 percentage points in $P(R^2 > 0.9)$ for reconstruction and 1.62 percentage points for generalization, plus a 0.0355 lower average DIV-diff RLMSE. The paper also introduces DIV-diff, a metric that compares the divergence of two vector fields over a dense grid rather than comparing a single rolled-out trajectory, which it argues is more stable and more comprehensive because it is independent of the initial condition.","feed_headline":"Derivative decoder adds 4.58-point gain in ODE recovery","feed_subtitle":"A transformer that also predicts derivatives beats prior symbolic ODE models and offers a more stable metric.","key_machinery":"The mechanism that carries the argument is the dual-decoder transformer: one decoder emits the symbolic ODE as a token sequence in prefix notation (with floats decomposed into sign, mantissa, and exponent tokens), and the other emits the derivative vector at each observed time point. A cross-entropy loss on each decoder is added, so the model is jointly trained to match the equation and its local behaviour. The companion evaluation object is DIV-diff, the root log mean squared error of the divergence (the trace of the Jacobian) of the predicted vector field versus the ground-truth field over a grid of points in the target region.","core_discovery":"The central discovery is that adding a derivative-prediction task to a symbolic ODE transformer improves both the accuracy and the stability of the recovered equations. DDOT's dual decoders produce the governing ODE in prefix notation and the derivative sequence $[f(x(t_i))]$ along the input trajectory; the total loss is the sum of the two cross-entropy losses, with the derivative term labelled a divergence-difference loss. Training this way makes the model approximate the true vector field pointwise rather than only reconstructing a single trajectory, and the paper reports consistent wins over ODEFormer, SINDy, and ProGED across five noise levels. The paper further claims that its DIV-diff metric, defined as the root logarithm mean squared error between the divergence fields of the predicted and true ODEs evaluated on a grid covering the target region, is a more reliable assessment of ODE recovery than single-trajectory reconstruction or generalization because it does not depend on the chosen initial point.","pith_inferences":["One implication the paper leaves implicit is that DIV-diff can be fooled by vector fields with matching divergence but different rotational or shear structure, so a stronger test would compare divergence together with, say, the vorticity or the full Jacobian.","The derivative decoder idea extends naturally to partial differential equations, where a decoder could predict the spatio-temporal derivative field and the symbolic PDE simultaneously.","A testable extension would be to train DDOT variants that predict derivatives at points outside the observed trajectory, making the auxiliary task even closer to the DIV-diff grid objective.","The paper's own data-generation pipeline could be reused to construct hard negative pairs for DIV-diff, probing whether the metric can distinguish systems that share divergence but differ in dynamics."],"forward_implications":["A derivative decoder can be added to any symbolic ODE transformer as an auxiliary training objective, improving recovery while leaving the inference-time process unchanged.","DIV-diff, if adopted as the community's evaluation metric, would shift ODE-discovery benchmarks away from initial-condition-dependent trajectory rollouts and toward whole-region comparisons of vector-field properties.","The anesthesia case study suggests the same architecture can produce interpretable, clinically-plausible ODEs from real patient monitoring data, not just simulated benchmarks.","The reported gains would make DDOT a strong default choice for symbolic ODE recovery when both equation structure and long-horizon trajectory fidelity matter."],"supporting_citations":[{"why":"Supplies the ODEFormer baseline and the ODEBench benchmark that all comparisons use.","marker":"[5]"},{"why":"Provides the end-to-end transformer symbolic regression training approach that the data generation and dual-decoder design follows.","marker":"[7]"},{"why":"Gives the float tokenization into sign, mantissa, and exponent components used in the symbolic decoder.","marker":"[3]"},{"why":"Provides the base sequence-to-sequence Transformer architecture that DDOT extends with a second decoder.","marker":"[15]"},{"why":"Supplies the numerical integration routine (solve_ivp) used to generate trajectory data from sampled ODEs.","marker":"[16]"}],"fun_headline_variants":["Dual decoder ODE transformer gains 4.6% recovery","Derivative task boosts ODE symbol recovery accuracy","New metric DIV-diff stabilizes ODE equation testing","Transformer with derivative decoder recovers ODEs better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The DIV-diff metric treats two ODEs as similar when their divergence fields match on a grid, which assumes divergence captures everything that matters about a dynamical system; two vector fields can share the same divergence yet produce very different trajectories.","fun_headline_variants_meta":{"raw":{"variants":["Dual decoder ODE transformer gains 4.6% recovery","Derivative task boosts ODE symbol recovery accuracy","New metric DIV-diff stabilizes ODE equation testing","Transformer with derivative decoder recovers ODEs better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1352,"prompt_tokens":994,"completion_tokens":358,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":292}},"tokens_in":610,"tokens_out":358,"duration_ms":4737,"temperature":1.0,"reasoning_tokens":292,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:16:03.022322+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute DIV-diff between $\\dot{x} = x, \\dot{y} = y$ and $\\dot{x} = x, \\dot{y} = y + x$ over a grid covering a neighbourhood of the origin: both have divergence 2, so DIV-diff would be near zero even though trajectories diverge quadratically in the second component, showing the metric can judge dynamically different ODEs as essentially identical.","supporting_citations":[{"cited_title":"In: ICLR","cited_arxiv_id":null,"evidence_quote":"Supplies the ODEFormer baseline and the ODEBench benchmark that all comparisons use."},{"cited_title":"In: NeurIPS (2022)","cited_arxiv_id":null,"evidence_quote":"Provides the end-to-end transformer symbolic regression training approach that the data generation and dual-decoder design follows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the float tokenization into sign, mantissa, and exponent components used in the symbolic decoder."},{"cited_title":"In: NIPS","cited_arxiv_id":null,"evidence_quote":"Provides the base sequence-to-sequence Transformer architecture that DDOT extends with a second decoder."},{"cited_title":"Nature Methods17, 261–272 (2020)","cited_arxiv_id":null,"evidence_quote":"Supplies the numerical integration routine (solve_ivp) used to generate trajectory data from sampled ODEs."}],"review_version":1}