{"id":"98598f46-fce0-4689-8fde-041561e65ce6","arxiv_id":"2506.11056","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An explainability method that instruments differentiable optimizers to emit natural language events and uses a language model to synthesize human-readable explanations of inverse problem optimization.","lead":"This paper proposes a method for explaining iterative optimization in inverse problems by converting numerical simulation traces into short natural language event descriptions, then using a language model to summarize the whole run. The authors test it on a railroad trajectory optimization problem and on training a small language model, showing that the generated explanations are informative to both language models and human raters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Explanations are evaluated for usefulness, not faithfulness; Section 3.6 even asserts a causal attribution the logged trace cannot support, so the central 'explainable' claim is not yet established.","rationale":"In good faith, the paper proposes a modular post hoc explanation pipeline and provides multiple converging evaluations. The user study and discrimination task give real evidence that natural-language explanations carry information, and the authors are honest about the reliance on user-defined transformation functions. I credit those points. The concern raised here is not that the optimization is wrong or that the approach is circular; it is that 'explanation' by construction must be faithful to the process being explained, and faithfulness is a distinct property from informativeness. The abstract's strongest claim is about enabling human-interpretable explanations; a fluent explanation that confidently asserts unsupported causality would actively mislead a domain expert, which is worse than no explanation. Section 3.6 provides the clearest instance: the framework as described captures parameter-change magnitudes and sample text, not attribution maps or causal effects, yet the text claims direct attribution. This is an internal evidence gap, not a disagreement with the field's consensus. The reader's weakest assumption, that the Phi functions preserve decision-relevant information, is related but distinct: even with perfect Phi, the LM synthesis step can be unfaithful. Therefore the conditional verdict remains appropriate, but the acceptance condition should explicitly require a faithfulness check rather than only stronger baselines or statistical reporting.","tokens_in":30053,"tokens_out":6657,"duration_ms":62252,"concrete_test":"Use the validation step from Section 3.6 and generate the step explanation under the true Phi_U update descriptions. Then regenerate it under a counterfactual trace in which the per-layer update descriptions are adversarially permuted (e.g., swap 'large change in LayerNorm weights' with 'no change in MLP weights') while keeping events, rewards, and generated samples fixed. If the explanation's causal attribution does not track the altered update descriptions, the LM is not reading the trace faithfully. Additionally, run the corresponding parameter ablation (zeroing or reversing that layer's update) to check whether the claimed behavioral change actually occurs; if it does not, the explanation is unfaithful.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that instrumenting an optimizer with natural-language events lets a domain expert obtain trustworthy explanations of an inverse-problem optimization. For that claim to hold, the LM explanations must be faithful to the actual trace: statements about why the optimizer moved control points, or which parameter updates caused a behavioral change, must be grounded in the logged events, rewards, and updates. None of the three evaluations measures faithfulness. The QA task (Section 3.3) shows above-chance obstacle identification; the discrimination task (Section 3.4) shows that explanations allow target/distractor separation; the user study (Section 3.5) shows that humans can match a description to the right visualization. All are utility or preference tests; a wrong explanation can pass all of them if it is plausible and correlated with the correct one. Section 3.6 illustrates the risk: the framework logs only average per-layer parameter changes and generated text samples, yet the stepwise explanation asserts that a qualitative leap was 'directly attributable to improved contextual integration and normalization ... as highlighted by our framework's attribution maps.' No attribution map, gradient, or ablation is described, and the loss decreased only marginally, so this causal claim is inferred by the LM from correlational update magnitudes. This is exactly the unfaithful-explanation failure mode that would mislead a domain expert. Section 4.1 acknowledges dependence on user-defined transformation functions but does not address LM faithfulness, so the gap is unmitigated. The paper should be accepted only if faithfulness is demonstrated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes xInv, a methodology for instrumenting differentiable simulators and optimizers so that they emit natural-language events, rewards, and parameter-update descriptions during forward and backward passes, and then uses a language model in a post-processing step to synthesize step-level and global explanations. The approach is demonstrated on a railroad-trajectory optimization problem with several standard optimizers, and on a small language-model training run, with evaluations consisting of an obstacle-removal question-answering task, a control-point discrimination task, and a human user study.","tokens_in":30282,"tokens_out":4214,"duration_ms":37512,"significance":"If the central claim holds, the paper offers a simple, modular bridge between numerical optimization and human-interpretable, queryable explanations, with broad potential applicability in scientific and engineering domains. The paper has clear strengths: it ships a concrete instantiation with a conversational agent, it evaluates across multiple LMs and with human raters, and the control-point discrimination and user-study results indicate that the generated descriptions carry usable information. However, the evidence currently supports usefulness and plausibility of explanations, not faithfulness to the underlying trace, and the LM-training example contains an unsupported causal attribution. The central idea is worth publishing once the faithfulness gap is addressed or the claims are appropriately scoped.","major_comments":[{"comment":"The stepwise example states: \"This qualitative leap was directly attributable to improved contextual integration and normalization, as highlighted by our framework's attribution maps.\" The method described in Section 2.2 and the supplemental logs record only validation samples, loss differences, and average per-layer parameter-magnitude changes; no attribution maps, gradients, or ablations are defined anywhere in the manuscript. This causal assertion is therefore not grounded in the logged trace, and it is exactly the kind of unsupported statement that could mislead a domain expert. Please either remove the causal and attribution language, or add a genuine attribution mechanism (e.g., per-layer gradient norms or counterfactual ablations) and verify that the explanation is supported by it.","section":"Section 3.6"},{"comment":"The three main evaluations are utility and discrimination tests: above-chance obstacle identification, target-versus-distractor matching, and human rating preference for the target visualization. Each of these can be passed by explanations that are plausible but not faithful to the logged trace, because a wrong explanation that is correlated with the correct one will still separate targets from distractors. Since the paper's central claim is about explainability, at least one evaluation should directly check faithfulness, for example by asking an LM or a human judge to verify individual explanation statements against the logged events, rewards, and updates, or by injecting an event that contradicts the explanation and measuring whether it is detected.","section":"Sections 3.3-3.5"},{"comment":"The question-answering evaluation lacks a baseline in which the LM is given the raw numerical trace (events, rewards, updates) instead of the natural-language transformation. As a result, the observed above-chance accuracy could be driven by the LM's prior knowledge or by the initial path events, rather than by the proposed natural-language explanation. The Numerical condition in Section 3.4 covers only control-point positions, not the full numerical trace. Please add a raw-trace condition, or an ablation that removes the natural-language descriptions, to isolate the contribution of the proposed transformation.","section":"Section 3.3"}],"minor_comments":[{"comment":"The formula t_m = (-v_m + sqrt(v_m^2 + 2 a_m s_m))/a_m is undefined when a_m = 0; please state the limiting case or otherwise handle this edge case in the simulation description.","section":"Section 2.1, Eq. (2)"},{"comment":"The user study reports a \"statistically significant preference\" for the target variant, but no test statistic, p-value, or effect size is reported; please specify the statistical test and the corresponding values.","section":"Section 3.5"},{"comment":"The discussion states that \"Section 3.1 demonstrates how LMs successfully interact with optimization systems,\" but Section 3.1 is the numerical optimization validation; the conversational interaction is in Section 3.2. Please correct the cross-reference.","section":"Section 4, first bullet"},{"comment":"The notation is inconsistent: Algorithm 1 uses \"amu\" and \"aeta\" for the frictional and air-resistance accelerations, while the text uses a_mu and a_eta; also the definition of E_k in Eq. (3) mixes superscripts and subscripts. Please unify the notation.","section":"Section 2.2 and Algorithm 1"},{"comment":"The supplemental material repeatedly refers to \"Section 4.2\" for details, but Section 4.2 in the main paper is the Conclusion; these references should point to the actual methodological sections.","section":"Supplemental Material"}],"recommendation":"major_revision","confidential_remarks":"The faithfulness concern in Section 3.6 is the main obstacle to acceptance; if the authors can add a genuine attribution mechanism or remove the causal overclaim, and add a raw-trace baseline in Section 3.3, the paper would be suitable for publication. The LM-training application is currently more of a qualitative teaser than a rigorous demonstration, and the title's 'Explainable' may overclaim unless faithfulness is directly evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read it. The core idea is actually new: instrument a differentiable simulator to emit natural-language events during forward and backward passes, then have an LM synthesize an explanation. I haven't seen that exact mechanism in the cited literature, and it's a sensible way to give domain experts a queryable interface to an optimizer. The modular design, with user-defined transformation functions and post hoc synthesis, is clean and easy to build on.\n\nWhat the paper does well: three independent evaluations all support the weaker claim that the explanations carry usable information. The QA task shows LMs can pick the most impactful obstacle to remove; the discrimination task is a clever way to test whether the explanation distinguishes the real optimized control points from noisy distractors; and the user study shows humans can match a description to the right visualization. That is real evidence, not just vibes. I also don't see circularity: independent LMs and humans judging LM-generated text is the right kind of external check.\n\nWhere the soft spots are, in order of seriousness. First, the stress-test note is right: the paper demonstrates utility, not faithfulness. None of the evaluations tests whether the explanation is actually true to the optimizer's internal reasoning. A plausible, correlated explanation could pass all three. Section 3.6 is the clearest case: the log contains average per-layer parameter changes and generated text samples, yet the generated explanation asserts a qualitative leap was \"directly attributable to improved contextual integration and normalization... as highlighted by our framework's attribution maps.\" No attribution map is described anywhere. That is the LM inventing a causal story from correlational update magnitudes, exactly the failure mode that would mislead someone. The limitations section acknowledges dependence on user-defined functions but never addresses LM faithfulness.\n\nSecond, the QA evaluation lacks a baseline where the LM sees the raw numerical trace, so we don't know how much the NL layer adds over raw numbers. Third, the user study says \"statistically significant\" without reporting a test statistic, p-value, or even the number of ratings per condition; the median 4-vs-2 split appears only in the top third of response time, which is a weak headline result. Fourth, no code or repository is shipped, despite detailed supplementary HTML. These are fixable, not fatal.\n\nOverall: this is a promising pipeline paper that should go to serious review. The authors need to either add a faithfulness check, such as counterfactual ablations of the logged events or comparing explanations to ground-truth optimizer decisions, or explicitly reframe the claim as \"useful, queryable summaries\" rather than causal explanations. I'd mention the missing raw-trace baseline and the user-study statistics in the review. For anyone working on XAI for optimization or LLM-simulator interfaces, this is worth reading and citing, but not yet worth leaning on for causal claims.","headline":"A genuinely new pipeline for NL explanations of optimizer behavior with real utility evidence, but the faithfulness gap highlighted in Section 3.6 is real and should gate the central claim.","tokens_in":30823,"tokens_out":2063,"would_cite":true,"duration_ms":20180,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An optimizer can explain itself if its simulator is instrumented to narrate forward and backward passes.","keywords":["inverse problems","explainable optimization","differentiable simulation","natural language explanations","language models","optimization traces","human-in-the-loop interpretability","post hoc explanation"],"falsifier":"Take a set of optimization instances and rerun the question-answering and discrimination evaluations with the reward transformation $\\Phi_R$ replaced by a constant mapping that never reports cost changes, while keeping the optimizer correct; if language models still identify the most beneficial obstacle removal at the accuracies reported in Section 3.3, the informative content is not actually coming from the trace, and if accuracy collapses, the assumption that the transformations must carry the information is confirmed.","tokens_in":29858,"feed_emoji":"🧠","tokens_out":8100,"duration_ms":63808,"temperature":0.7,"pith_summary":"Inverse problems—estimating inputs so a known simulator produces a desired output—are usually solved by iterative optimization, but that optimization loop is hard for domain experts to read. This paper proposes xInv, a methodology that wraps a differentiable optimizer with a natural-language layer: the simulator emits events during its forward and backward passes, these events are converted into qualitative descriptions, and a language model assembles them into explanations. The authors argue that this simple instrumentation turns a cryptic optimization trace into a queryable narrative, and they demonstrate it on a railroad-trajectory design problem and on the training of a small language model. If correct, it gives domain experts a way to ask why an optimizer behaved as it did without reading raw numbers.","feed_headline":"Optimizers can narrate their own runs in plain language","feed_subtitle":"A language model turns the narrated optimization into answers a domain expert can act on.","key_machinery":"The load-bearing mechanism is the three-phase trace pipeline: emission, transformation, and synthesis. At optimization step $k$ the optimizer emits events $E_k$, rewards $R_k$, and updates $U_k$; the transformation functions map those numerical quantities to qualitative natural-language descriptions whose magnitudes are scaled relative to the average change seen across the run; and a language model combines them first into per-step descriptions and then into a global narrative that includes reward-change information. Event and update sample rates control how fine-grained the trace is. The same instrumentation is applied to LM training by treating generated samples, losses, and per-layer parameter changes as the raw signals.","core_discovery":"The paper's central claim is that the iterative optimization of inverse problems can be made explainable while keeping the underlying optimizer essentially traditional, by instrumenting it to emit signals. At each step the optimizer emits events, rewards, and parameter updates, and domain-defined transformation functions $\\Phi_E$, $\\Phi_R$, and $\\Phi_U$ convert these numerical signals into natural-language statements such as \"Change in acceleration: small (positive)\" and \"Control point 1: Magnitude: very small (ESE).\" A language model then reads the resulting trace and produces step-level descriptions plus a global narrative, and users can ask follow-up questions interactively. The two demonstrations—a physics-based railroad optimization and the training of a small LM on physics question answering—are meant to show that the method works for both white-box and black-box systems.","pith_inferences":["If this pipeline is sound, the practical bottleneck shifts from optimizer design to trace design: the fidelity of any explanation is bounded by what the domain expert chooses to expose through $\\Phi_E$, $\\Phi_R$, and $\\Phi_U$.","The same idea could be used as an audit log for high-stakes optimization decisions, since it converts a numerical history into a reviewable narrative that a human can inspect after the fact.","A natural stress test would be to run the obstacle-identification task with deliberately impoverished transformation functions to see how much information loss the language model can tolerate before decisions degrade.","The method is likely to compose with other explainability tools, making the natural-language trace a reusable interface for attribution maps, sensitivity analysis, or policy-capturing techniques."],"forward_implications":["A domain expert can converse with an optimizer: asking for a speed-prioritized versus cost-prioritized run, requesting state changes, and querying why one route is faster or cheaper.","Without changing the underlying optimizer, the generated explanations let a language model identify which obstacle removal would most improve the result, with accuracy well above random chance.","Explanations carry enough information for a language model to distinguish optimized control points from noisy distractors even at low noise levels, with performance approaching the upper bound once noise grows.","Human participants rate the explanation as matching the true optimization significantly more than a distractor, and the preference strengthens for respondents who take longer to read.","Because only the emission and transformation functions are domain-specific, the same method transfers from a physics simulator to a black-box neural-network training run."],"supporting_citations":[{"why":"Documents that language models are unreliable at interpreting raw simulation traces containing numerical values, motivating the natural-language instrumentation.","marker":"[9]"},{"why":"Earlier finding that language models struggle to infer physical-system parameters from traces, supporting the need for explicit event emission.","marker":"[37]"},{"why":"Supplies the reasoning-and-acting loop used to let the language model run optimizations, observe the narrated results, and answer user questions.","marker":"[54]"},{"why":"Provides the declarative prompting pipeline used to implement the explanation and question-answering signatures.","marker":"[26]"},{"why":"Provides the style of reasoning-in-steps prompting used by the explanation generator.","marker":"[51]"},{"why":"Quantifies language models' numerical-understanding limits, supporting the design choice of grid rather than floating-point inputs.","marker":"[53]"}],"fun_headline_variants":["Inverse problems get a plain-English narrator","Optimization traces become natural-language stories","Language model narrates optimizer runs for domain experts","Explainable inverse problems via narrated optimizer runs","Optimizer calls out its own steps in human language"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the numerical-to-language transformation functions defined by a domain expert preserve the decision-relevant information; if they do not, the explanations can be uninformative or misleading even when the optimization itself is sound.","fun_headline_variants_meta":{"raw":{"variants":["Inverse problems get a plain-English narrator","Optimization traces become natural-language stories","Language model narrates optimizer runs for domain experts","Explainable inverse problems via narrated optimizer runs","Optimizer calls out its own steps in human language"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000367,"raw_usage":{"total_tokens":1919,"prompt_tokens":837,"completion_tokens":1082,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":1014}},"tokens_in":453,"tokens_out":1082,"duration_ms":8343,"temperature":1.0,"reasoning_tokens":1014,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:55:09.994762+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of optimization instances and rerun the question-answering and discrimination evaluations with the reward transformation $\\Phi_R$ replaced by a constant mapping that never reports cost changes, while keeping the optimizer correct; if language models still identify the most beneficial obstacle removal at the accuracies reported in Section 3.3, the informative content is not actually coming from the trace, and if accuracy collapses, the assumption that the transformations must carry the information is confirmed.","supporting_citations":[{"cited_title":"Simlm: Can language models infer parameters of physical systems?, 2024","cited_arxiv_id":null,"evidence_quote":"Earlier finding that language models struggle to infer physical-system parameters from traces, supporting the need for explicit event emission."},{"cited_title":"Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts","cited_arxiv_id":null,"evidence_quote":"Provides the declarative prompting pipeline used to implement the explanation and question-answering signatures."},{"cited_title":"Number cookbook: Num- ber understanding of language models and how to improve it","cited_arxiv_id":null,"evidence_quote":"Quantifies language models' numerical-understanding limits, supporting the design choice of grid rather than floating-point inputs."}],"review_version":1}