{"id":"e352275b-5b41-4b4a-a24b-2799a57078e6","arxiv_id":"2605.31064","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"DCRC applies data-centric methods with adversarial examples and program synthesis to produce verifiable reasoning programs for financial question answering.","lead":"The paper proposes a Data-centric Reasoning Compiler (DCRC) that uses adversarial data construction, multi-stage training of a structuring agent, and compile-and-execute inference to reduce numerical hallucinations in financial QA systems. A smart generalist might read it because reliable numerical reasoning in AI is essential for trustworthy financial services and other high-stakes decision systems.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader correctly flagged the generalization step as the key empirical hinge. Full-text description of the three phases does not introduce a stronger internal flaw; the 'by design' phrasing is therefore best read as an empirical claim rather than a formal guarantee. No adjustment to UNVERDICTED is warranted.","tokens_in":1710,"tokens_out":276,"duration_ms":11333,"concrete_test":"Re-run the real-world online deployment evaluation (mentioned in the abstract) with an added metric: fraction of DSA-generated programs that fail to execute or produce incorrect numerical results on a fresh sample of 200 live user queries; compare against the offline benchmark numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DCRC 'ensures faithful numerical reasoning by design' is supported by the three-phase construction (adversarial synthesis + DSA training + compile-and-execute). The weakest assumption identified by the reader—generalization of the DSA from synthesized data to real queries—is a standard empirical risk rather than an internal inconsistency in the argument. The paper reports offline benchmark results plus a real-world deployment, which directly addresses that assumption. No hidden circularity, unstated assumption in the program synthesis step, or missing verification of the execution engine is apparent from the described pipeline.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents the Data-centric Reasoning Compiler (DCRC), a data-centric framework designed to mitigate numerical hallucinations in LLMs for financial question answering. The framework is built around three phases: (1) adversarial data construction that synthesizes examples with controlled noise to promote robustness, (2) multi-stage training of a Data-centric Structuring Agent (DSA) that performs explicit evidence auditing and synthesizes executable programs, and (3) a compile-and-execute inference process that converts user queries and retrieved documents into verifiable reasoning programs. The authors assert that this approach ensures faithful numerical reasoning by design and provide validation through experiments on established offline benchmarks as well as a real-world online financial QA system deployment.","tokens_in":1785,"tokens_out":374,"duration_ms":53264,"significance":"If the results hold, the significance is high as it offers an integrated solution to the challenges of noise sensitivity, calculation fragility, and auditability in RAG-based systems for high-stakes domains. The data-centric paradigm, with its emphasis on synthesized adversarial data and program synthesis, provides a structured alternative to model-centric optimizations. The real-world deployment directly tests the generalization of the DSA, addressing the primary empirical risk in the approach.","major_comments":[],"minor_comments":[{"comment":"The abstract would be strengthened by including specific performance metrics or improvements from the experiments to better support the claims of effectiveness.","section":"Abstract"},{"comment":"Ensure all acronyms (e.g., RAG, FinQA, DSA, DCRC) are defined at first use in the main text.","section":null},{"comment":"The multi-stage training process for the DSA could benefit from a diagram or pseudocode to clarify the stages and their objectives.","section":"Method"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive summary, high significance assessment, and recommendation of minor revision. We are pleased that the data-centric approach and real-world deployment are viewed favorably.","responses":[],"tokens_in":1284,"tokens_out":54,"duration_ms":11120,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The key takeaway is that the authors propose DCRC, which fights numerical hallucinations in financial QA by synthesizing adversarial training data, training a Data-centric Structuring Agent to audit evidence and synthesize programs, and then using a compile-and-execute step for inference. This data-centric shift aims to make reasoning faithful by construction rather than hoping the LLM gets the numbers right.\n\nWhat stands out as new is the cohesive three-phase framework that handles the RAG challenges of noise, fragile calculations, and lack of auditability in one system. They move away from model-centric tweaks to retrievers or generators alone. The paper does well in describing how the adversarial construction creates controlled noise to build robustness, and in validating the whole thing not just on offline benchmarks but through actual deployment in an online financial QA system. That deployment step is useful because it tests generalization to real queries.\n\nOn the soft spots, the approach depends heavily on the synthesized data covering the kinds of issues that appear in practice. The weakest assumption is that the DSA will produce correct programs for unseen real-world cases without introducing its own errors. The real-world results help mitigate this concern, but more detail on error analysis or cases where it still fails would make the claims stronger. There's also the question of how the execution engine is implemented and verified, though the paper seems to treat that as given.\n\nThis work is for people focused on making LLM-based systems reliable in finance or other domains where numbers matter. A reader looking for practical ways to add verifiability to RAG pipelines would find it relevant. It deserves a serious referee given the integrated approach and the live system evidence.\n\nI would recommend sending it for peer review.","headline":"DCRC reframes numerical hallucinations in FinQA as a compilation task using adversarial data synthesis and a structuring agent that outputs executable programs, with benchmark and deployment results.","tokens_in":2261,"tokens_out":416,"would_cite":false,"duration_ms":23520,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A data-centric compilation framework combats numerical hallucinations in financial QA by synthesizing adversarial data and compiling queries into verifiable executable programs.","keywords":["financial question answering","numerical hallucinations","data-centric compilation","retrieval-augmented generation","program synthesis","adversarial data construction","LLM reasoning robustness"],"falsifier":"A deployed query where the DSA outputs an executable program that yields an incorrect numerical result despite the retrieved documents containing the correct supporting figures.","tokens_in":2612,"feed_emoji":"","tokens_out":615,"duration_ms":18114,"temperature":0.7,"pith_summary":"The paper establishes that existing model-centric fixes for retrieval-augmented generation fail to jointly solve noise sensitivity, calculation fragility, and auditability in financial question answering. It advances a data-centric alternative called the Data-centric Reasoning Compiler that first builds training examples with controlled noise, then trains a Data-centric Structuring Agent to audit evidence and synthesize programs, and finally executes those programs at inference time. A sympathetic reader would care because this shifts reliability from opaque model improvements to an explicit, checkable compilation process. If the approach holds, numerical answers in high-stakes financial systems become faithful by construction rather than by chance or post-processing.","feed_headline":"Data-centric compiler turns financial queries into verifiable programs","feed_subtitle":"Adversarial training data and an agent for evidence auditing and program synthesis address noise, fragility, and auditability in RAG-based F","key_machinery":"The Data-centric Structuring Agent (DSA) that performs explicit evidence auditing and synthesizes executable reasoning programs from queries and documents.","core_discovery":"The Data-centric Reasoning Compiler (DCRC) works through adversarial data construction that injects controlled noise, multi-stage training of a Data-centric Structuring Agent capable of explicit evidence auditing and program synthesis, and a compile-and-execute inference step that converts user queries plus retrieved documents into verifiable reasoning programs, thereby ensuring faithful numerical reasoning by design.","pith_inferences":["The same data-synthesis-plus-compilation pattern could be tested on non-financial domains that require precise numerical or logical steps.","Success would imply that robustness gains can come from curated training distributions rather than solely from larger generator models.","One could measure whether the DSA's program synthesis scales to queries needing more than two or three arithmetic operations."],"forward_implications":["Training on adversarial data with controlled noise reduces sensitivity to retrieval errors.","Converting reasoning into executable programs overcomes calculation fragility in LLMs.","Explicit program synthesis provides an auditable trace that addresses the auditability crisis.","The same pipeline shows gains on offline benchmarks and in a live online financial QA deployment."],"fun_headline_variants":["Data-centric compiler addresses hallucinations via program synthesis","DSA agent enables verifiable reasoning programs for financial QA","Adversarial construction trains robust FinQA reasoning agent","Framework compiles queries into executable financial reasoning programs"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The synthesized adversarial examples and the trained DSA will generalize to real user queries and retrieved documents while always producing correct executable programs.","fun_headline_variants_meta":{"raw":{"variants":["Data-centric compiler addresses hallucinations via program synthesis","DSA agent enables verifiable reasoning programs for financial QA","Adversarial construction trains robust FinQA reasoning agent","Framework compiles queries into executable financial reasoning programs"]},"model":"grok-4.3","cost_usd":0.004729,"raw_usage":{"total_tokens":2338,"prompt_tokens":678,"num_sources_used":0,"completion_tokens":56,"cost_in_usd_ticks":47287000,"prompt_tokens_details":{"text_tokens":678,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1604,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":678,"tokens_out":56,"duration_ms":12576,"temperature":1.0,"reasoning_tokens":1604,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T21:10:16.750774+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A deployed query where the DSA outputs an executable program that yields an incorrect numerical result despite the retrieved documents containing the correct supporting figures.","supporting_citations":[],"review_version":1}