{"id":"8f6802ec-c64c-45eb-a20f-3e28f29182e4","arxiv_id":"2605.31238","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Graph-constrained path selection on contextual keyword centroids decouples path enumeration from teacher verbalization to scale multi-hop QA data from plain text, enabling 4.4x corpus expansion and raising closed-book Token F1 from 21.66% to 38.58% on CUAD legal contracts.","lead":"This paper decouples reasoning path discovery from question-answer verbalization by building an offline graph of keyword centroids and applying geometric constraints to select valid paths before invoking a teacher model. If effective, the approach could help generate large volumes of multi-hop training data for domain-specific documents where such data is scarce.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Gram-matrix justification for the five constraints may not ensure they select only teacher-verbalizable paths without bias or exclusion in dense boilerplate regions.","rationale":"The ablation result directly supports the scaling interpretation only if the constraints function as claimed without introducing the very bias or exclusion the Gram-matrix analysis is meant to rule out; this is the same load-bearing assumption the reader identified, and the abstract provides no additional evidence that would resolve it.","tokens_in":1831,"tokens_out":331,"duration_ms":18542,"concrete_test":"Extract the actual sentence embeddings and Gram matrices from the CUAD corpus sections used for path enumeration; recompute the observed maximum endpoint drift angles and clique exit behavior with/without the upper similarity bound; verify whether the five constraints change the fraction of paths the teacher successfully verbalizes by >10% relative to an unconstrained enumeration on the same documents.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that gains come from 4.4× corpus expansion rather than per-chain quality—rests on the matched-scale ablation showing indistinguishable downstream performance for constrained vs. unconstrained chains. This interpretation requires that the offline graph enumeration with its five geometric admissibility constraints (justified by Gram-matrix arguments on local similarity bounds permitting ~91° endpoint drift and necessitating an upper bound to exit dense cliques) correctly identifies paths the teacher can verbalize without systematic bias or omission of valid multi-hop chains. The arguments assume embedding geometry that may fail to hold for repetitive legal templates in CUAD, where local similarity alone could still permit drift or clique entrapment that affects synthesizability.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents a method for scaling multi-hop QA training data from unannotated specialized text by enumerating paths offline on a graph of contextual keyword centroids subject to five geometric admissibility constraints, with a teacher model used only for verbalization of validated paths. Gram-matrix arguments are provided to justify the constraints by showing that local similarity alone allows up to ~91° endpoint drift and that an upper bound is needed to escape dense cliques from boilerplate. A matched-size ablation demonstrates that constrained and unconstrained chains perform similarly at equal scale, attributing the overall gain to a 4.4× increase in usable data volume. Fine-tuning Qwen3-32B on 80K examples from the CUAD corpus improves closed-book Token F1 from 21.66% to 38.58%. Code is released.","tokens_in":1994,"tokens_out":507,"duration_ms":23140,"significance":"If the central claim holds—that the constraints primarily increase the number of teacher-verbalizable paths without altering per-path quality or introducing bias—this work could meaningfully advance data construction for compositional reasoning in domain-specific corpora characterized by repetitive structures. The explicit ablation isolating volume from quality is a strength, as is the public release of the code, which supports reproducibility. The approach reframes graph-based constraints as a scalability tool rather than a quality filter.","major_comments":[{"comment":"Methods section, Gram-matrix arguments for the five constraints: the claim that these constraints raise teacher synthesizability without systematic bias or exclusion of valid multi-hop chains in dense boilerplate regions of CUAD is load-bearing for interpreting the 4.4× expansion as the source of gains. The ~91° drift bound is derived, but no direct evidence (e.g., verbalization success rates on selected vs. rejected paths or manual inspection in template-heavy sections) is shown to confirm the constraints avoid bias or omission in repetitive legal embeddings.","section":"Methods (Gram-matrix arguments)"},{"comment":"Experiments section, matched-size ablation: the report that constrained and unconstrained chains yield indistinguishable downstream performance at equal scale underpins the volume-over-quality interpretation. However, the construction of the unconstrained set (sampling method, path enumeration without graph, and distribution matching to the constrained set) requires explicit detail to rule out selection effects that could confound the ablation result.","section":"Experiments (matched-size ablation)"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and positive assessment of the work's significance. We address each major comment below and will revise the manuscript to incorporate the requested clarifications and evidence.","responses":[{"response":"We agree that direct empirical evidence would strengthen the load-bearing claim. The Gram-matrix arguments establish theoretical bounds on drift and clique escape, while the matched-size ablation supports that per-path quality is not systematically altered. In the revision we will add verbalization success rates comparing constrained versus rejected paths and qualitative inspection of examples from template-heavy CUAD sections.","revision_made":"yes","referee_comment":"[Methods (Gram-matrix arguments)] the claim that these constraints raise teacher synthesizability without systematic bias or exclusion of valid multi-hop chains in dense boilerplate regions of CUAD is load-bearing... no direct evidence (e.g., verbalization success rates on selected vs. rejected paths or manual inspection in template-heavy sections) is shown to confirm the constraints avoid bias or omission in repetitive legal embeddings."},{"response":"We will expand the Experiments section with explicit details on the unconstrained-set construction, including the path-enumeration procedure without graph constraints, the sampling method, and the exact distribution-matching criteria used to ensure the sets are comparable at equal scale.","revision_made":"yes","referee_comment":"[Experiments (matched-size ablation)] the construction of the unconstrained set (sampling method, path enumeration without graph, and distribution matching to the constrained set) requires explicit detail to rule out selection effects that could confound the ablation result."}],"tokens_in":1601,"tokens_out":344,"duration_ms":15117,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's central result is that decoupling path enumeration from verbalization lets them scale multi-hop QA data 4.4 times on the CUAD legal corpus. They build a graph over contextual keyword centroids, apply five geometric admissibility constraints justified by Gram-matrix arguments on similarity bounds and endpoint drift, then have the teacher only verbalize the accepted paths. A matched-size ablation finds no downstream difference between constrained and unconstrained chains, so the lift on Qwen3-32B (21.66% to 38.58% closed-book Token F1) comes from having more examples rather than higher-quality ones per example.\n\nWhat is new is the explicit offline graph step plus the Gram-matrix framing that local similarity alone permits ~91° drift and that an upper bound is needed to escape boilerplate cliques. The ablation cleanly isolates volume as the mechanism, and they release the code.\n\nThe soft spot is whether the five constraints reliably select only paths the teacher can verbalize without systematic omission or bias in dense embedding regions. The abstract states the geometric argument, but the precise constraint definitions, centroid construction, and any filtering rules are not shown, so it is difficult to judge how well the geometry holds for repetitive legal templates. The result is also reported on a single corpus and model.\n\nThis is for practitioners who need to generate compositional training data from plain specialized text. It deserves peer review because the method is concrete, the ablation addresses the scaling claim directly, and the performance delta is measurable.","headline":"They get 4.4x more multi-hop examples from legal text by offline graph path selection with five geometric constraints, and the ablation shows the gain is volume not quality.","tokens_in":2527,"tokens_out":383,"would_cite":false,"duration_ms":15763,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Graph constraints on reasoning paths expand usable multi-hop training data by 4.4 times without improving per-path quality.","keywords":["multi-hop reasoning","training data generation","graph constraints","path selection","compositional reasoning","language model fine-tuning","embedding similarity","data scaling"],"falsifier":"A matched-scale experiment in which constrained paths produce measurably higher downstream performance than unconstrained paths, or a full-scale run showing no gain from the 4.4 times larger corpus.","tokens_in":2725,"feed_emoji":"","tokens_out":669,"duration_ms":25375,"temperature":0.7,"pith_summary":"The paper shows that multi-hop reasoning data can be scaled from unstructured text by first building a graph of contextual keyword centroids and enumerating admissible paths using five geometric constraints before asking a teacher model to turn those paths into question-answer pairs. This approach avoids the degradation seen in joint discovery and verbalization methods when dealing with repetitive or densely cross-referenced documents. Experiments demonstrate that the main benefit is a 4.4 times larger usable corpus, leading to improved performance after fine-tuning. At matched training sizes, constrained and unconstrained paths perform similarly, indicating the constraints primarily increase the number of synthesizable examples rather than their individual quality.","feed_headline":"Graph constraints expand multi-hop data 4.4 times","feed_subtitle":"The gain comes from more usable examples rather than better quality per chain, improving closed-book performance on specialized documents.","key_machinery":"The graph of contextual keyword centroids with five geometric admissibility constraints that enforce local similarity bounds to prevent endpoint drift and escape dense embedding cliques.","core_discovery":"The central discovery is that enforcing five geometric admissibility constraints on paths through a graph of contextual keyword centroids allows enumeration of 4.4 times more verbalizable multi-hop chains from plain text than unconstrained methods, with the performance gain arising from corpus expansion rather than superior chain quality, as shown by matched-size ablations where both yield indistinguishable results.","pith_inferences":["The same decoupling of enumeration and verbalization could apply to other domains with repetitive text structures.","Relaxing one or more of the five constraints might alter the expansion factor and offer a tunable parameter for data volume.","The result suggests that quantity of examples can outweigh per-example sophistication in multi-hop fine-tuning regimes.","Similar graph-based filtering might help scale other forms of synthetic training data where joint generation fails."],"forward_implications":["Larger volumes of multi-hop data become feasible from specialized corpora without manual curation.","Teacher models can be invoked only on pre-validated paths, increasing the fraction of synthesizable examples.","Performance on compositional reasoning tasks improves through data volume rather than path quality.","The constraints function to raise teacher synthesizability rather than to select higher-quality chains.","At equal training scale the two approaches produce equivalent results on downstream tasks."],"fun_headline_variants":["Graph constraints scale multi-hop chains 4.4x","Geometric constraints yield 4.4x more training chains","Keyword centroid graphs scale multi-hop data 4.4x","Path graph constraints expand chain count 4.4 times"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The five geometric constraints correctly identify paths that the teacher model can verbalize without introducing systematic bias or excluding valid chains in dense embedding regions formed by boilerplate text.","fun_headline_variants_meta":{"raw":{"variants":["Graph constraints scale multi-hop chains 4.4x","Geometric constraints yield 4.4x more training chains","Keyword centroid graphs scale multi-hop data 4.4x","Path graph constraints expand chain count 4.4 times"]},"model":"grok-4.3","cost_usd":0.009114,"raw_usage":{"total_tokens":4122,"prompt_tokens":738,"num_sources_used":0,"completion_tokens":59,"cost_in_usd_ticks":91137000,"prompt_tokens_details":{"text_tokens":738,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3325,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":738,"tokens_out":59,"duration_ms":23039,"temperature":1.0,"reasoning_tokens":3325,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T22:23:48.596336+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A matched-scale experiment in which constrained paths produce measurably higher downstream performance than unconstrained paths, or a full-scale run showing no gain from the 4.4 times larger corpus.","supporting_citations":[],"review_version":1}