{"id":"6590f2ac-41c4-42cd-a2ac-697eec2f7feb","arxiv_id":"2605.24454","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"DaR integrates question decomposition with parametric query refinement to improve retrieval accuracy and answer quality on the KoBLEX multi-hop legal QA benchmark.","lead":"The paper proposes Decompose-and-Refine (DaR), a framework that breaks complex legal questions into atomic sub-questions and creates statute-aligned queries to retrieve relevant legal provisions. A smart generalist might read it because reliable, traceable answers in legal AI could reduce errors in high-stakes domains like statutory interpretation.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's identification of the vocabulary-gap / hallucination assumption matches the paper's own framing of the problem and proposed solution. Because the full text is stipulated to be readable and no contradictory or under-supported element surfaces in the abstract-level claim, the provisional UNVERDICTED stance does not require adjustment.","tokens_in":1749,"tokens_out":255,"duration_ms":30688,"concrete_test":"Re-run the KoBLEX retrieval and answer-quality experiments from the full paper's §4 using the exact same Qwen3-32B and Gemma3-27B checkpoints and prompts; confirm that the reported deltas versus baselines remain within 2% of the published numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract states that DaR improves retrieval accuracy and answer quality on KoBLEX with two models. The reader's weakest assumption correctly flags the core mechanism (decomposition + parametric refinement closing the vocabulary gap without added hallucination). With the full manuscript now available for inspection, no internal inconsistency, missing control, or unsupported leap in the central claim is evident from the provided description; the method is presented as directly addressing the stated problem with reported gains.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces Decompose-and-Refine (DaR), a statute-grounded framework for legal question answering that progressively decomposes complex multi-hop questions into atomic sub-questions and generates statute-aligned parametric queries to retrieve the most central statutory provision per legal issue. It is evaluated on the KoBLEX Korean multi-hop LQA benchmark using Qwen3-32B and Gemma3-27B, claiming consistent improvements in retrieval accuracy and final answer quality over existing approaches while enabling transparent issue-level verification of reasoning.","tokens_in":1818,"tokens_out":316,"duration_ms":37853,"significance":"If the empirical gains hold, the tight integration of decomposition with parametric refinement offers a targeted way to close the vocabulary gap between natural-language questions and statutory text, potentially lowering hallucination risk in multi-hop statutory LQA and improving verifiability. The approach is a direct response to limitations in prior multi-hop QA methods that rely on natural-language reasoning without explicit query reformulation.","major_comments":[{"comment":"Abstract: the central claim that 'DaR consistently improves both retrieval accuracy and final answer quality' is asserted without any quantitative deltas, baseline names, metric definitions, or error analysis, which is load-bearing for evaluating the empirical contribution.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract and introduction would benefit from a one-sentence definition or citation for 'parametric knowledge-based query refinement' on first use.","section":null},{"comment":"Consider adding a short illustrative example of a decomposed sub-question and its corresponding parametric query in §3 to clarify the refinement step.","section":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive evaluation and the specific suggestion regarding the abstract. The comment identifies a clear opportunity to strengthen the presentation of our empirical results, and we will address it directly in revision.","responses":[{"response":"We agree that the abstract would be strengthened by including concrete quantitative support for the central claim. In the revised manuscript we will update the abstract to report the specific improvements observed (e.g., retrieval accuracy gains and answer-quality metrics) together with the names of the primary baselines, the evaluation metrics used, and a brief indication of the scale of the gains. This change will make the empirical contribution immediately verifiable from the abstract while preserving the existing technical description of the framework.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central claim that 'DaR consistently improves both retrieval accuracy and final answer quality' is asserted without any quantitative deltas, baseline names, metric definitions, or error analysis, which is load-bearing for evaluating the empirical contribution."}],"tokens_in":1322,"tokens_out":226,"duration_ms":11396,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core idea here is straightforward: decompose a multi-hop legal question step by step, then use the model's parametric knowledge to rewrite each piece as a query that better matches statutory language. This targets the vocabulary mismatch that standard retrieval often misses in legal text.\n\nThe paper does a clean job of making the process explicit. By tying each sub-question to one central statutory provision, it produces traceable outputs that line up with how lawyers actually check reasoning. The evaluation on KoBLEX with Qwen3-32B and Gemma3-27B shows gains in both retrieval accuracy and final answer quality over prior methods, and the transparency angle is a practical plus for the domain.\n\nThe main limitation is that the abstract gives no numbers, deltas, or error breakdowns, so the actual size of the improvement is hard to gauge without the full tables. The method also assumes the parametric refinement step stays reliable and does not add hallucinations; the results would need to demonstrate that clearly. Testing is limited to Korean statutes, so transfer to other languages or legal systems remains open.\n\nThis is aimed at people working on grounded legal QA or multi-hop retrieval where traceability matters. A reader who needs a concrete structure for statute-level grounding will find usable pieces here.\n\nIt deserves peer review. The approach is well-motivated and the benchmark is appropriate; the experiments just need the details filled in for a full assessment.","headline":"DaR breaks legal questions into sub-questions then refines each into statute-specific parametric queries, which improves retrieval and answer quality on KoBLEX with the two tested models.","tokens_in":2284,"tokens_out":361,"would_cite":false,"duration_ms":26416,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Decompose-and-Refine improves retrieval accuracy and answer quality in multi-hop legal question answering by breaking down questions and refining queries to match statutory text.","keywords":["legal question answering","multi-hop reasoning","statutory retrieval","query refinement","question decomposition","LLM","Korean legal benchmark"],"falsifier":"Running the same models on a second multi-hop legal benchmark in a different language and finding no rise in retrieval precision or a rise in answers lacking statute support.","tokens_in":2654,"feed_emoji":"⚖️","tokens_out":594,"duration_ms":20265,"temperature":0.7,"pith_summary":"The paper presents Decompose-and-Refine as a way to handle complex legal questions that span multiple statutes. It splits each question into simpler sub-questions and then rewrites retrieval queries using the model's built-in knowledge so they align better with formal legal wording. This targets the mismatch in language that often causes retrieval to miss the right provisions. Tests on a Korean statutory benchmark with two large models show gains in both finding relevant laws and producing grounded answers. The structure also lets each legal issue be checked against its specific supporting text.","feed_headline":"Decomposition plus refinement raises legal QA retrieval accuracy","feed_subtitle":"Breaking questions into sub-parts and aligning each to statutes cuts errors on multi-hop statutory tasks.","key_machinery":"The Decompose-and-Refine (DaR) framework that combines step-wise question decomposition with parametric knowledge-based query refinement to retrieve one central statutory provision per legal issue.","core_discovery":"DaR progressively decomposes a complex legal question into atomic sub-questions and generates statute-aligned parametric queries for each sub-question, enabling the selection of a single most central statutory provision corresponding to each legal issue. This integration of decomposition and parametric refinement addresses the vocabulary gap in statutory LQA.","pith_inferences":["The same split-and-refine pattern could reduce vocabulary mismatch in other retrieval-heavy domains such as medical guidelines.","Issue-by-issue statute links could support automated checks for whether an answer stays within cited authorities.","Parametric refinement might lower the need for very large external retrieval indexes in statutory tasks."],"forward_implications":["Retrieval of relevant statutes becomes more accurate for questions needing several legal provisions.","Final answers gain quality because each sub-question links to an explicit statute.","Complex reasoning steps become verifiable at the level of individual legal issues.","The gains hold for Qwen3-32B and Gemma3-27B on the KoBLEX benchmark."],"fun_headline_variants":["DaR decomposes questions to refine legal statute retrieval","Parametric queries after decomposition aid multi-hop legal QA","Sub-question breakdown enables central statute selection in LQA","DaR tackles vocabulary gap with stepwise decomposition and refinement"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Step-wise decomposition together with the model's own knowledge for query rewriting will close the wording gap between user questions and statutes enough to raise retrieval accuracy without raising unsupported answers.","fun_headline_variants_meta":{"raw":{"variants":["DaR decomposes questions to refine legal statute retrieval","Parametric queries after decomposition aid multi-hop legal QA","Sub-question breakdown enables central statute selection in LQA","DaR tackles vocabulary gap with stepwise decomposition and refinement"]},"model":"grok-4.3","cost_usd":0.004677,"raw_usage":{"total_tokens":2241,"prompt_tokens":687,"num_sources_used":0,"completion_tokens":61,"cost_in_usd_ticks":46765500,"prompt_tokens_details":{"text_tokens":687,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1493,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":687,"tokens_out":61,"duration_ms":17520,"temperature":1.0,"reasoning_tokens":1493,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T13:20:37.029981+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the same models on a second multi-hop legal benchmark in a different language and finding no rise in retrieval precision or a rise in answers lacking statute support.","supporting_citations":[],"review_version":1}