Pith. sign in

REVIEW 1 major objections 2 minor 55 references

Decompose-and-Refine improves retrieval accuracy and answer quality in multi-hop legal question answering by breaking down questions and refining queries to match statutory text.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 13:20 UTC pith:K4L5ZXOO

load-bearing objection DaR breaks legal questions into sub-questions then refines each into statute-specific parametric queries, which improves retrieval and answer quality on KoBLEX with the two tested models. the 1 major comments →

arxiv 2605.24454 v1 pith:K4L5ZXOO submitted 2026-05-23 cs.CL

Decompose-and-Refine: Structured Legal Question Answering with Parametric Retrieval

Jihyung lee , Hyounghun Kim , Gary Lee This is my paper
classification cs.CL
keywords legal question answeringmulti-hop reasoningstatutory retrievalquery refinementquestion decompositionLLMKorean legal benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents Decompose-and-Refine as a way to handle complex legal questions that span multiple statutes. It splits each question into simpler sub-questions and then rewrites retrieval queries using the model's built-in knowledge so they align better with formal legal wording. This targets the mismatch in language that often causes retrieval to miss the right provisions. Tests on a Korean statutory benchmark with two large models show gains in both finding relevant laws and producing grounded answers. The structure also lets each legal issue be checked against its specific supporting text.

Core claim

DaR progressively decomposes a complex legal question into atomic sub-questions and generates statute-aligned parametric queries for each sub-question, enabling the selection of a single most central statutory provision corresponding to each legal issue. This integration of decomposition and parametric refinement addresses the vocabulary gap in statutory LQA.

What carries the argument

The Decompose-and-Refine (DaR) framework that combines step-wise question decomposition with parametric knowledge-based query refinement to retrieve one central statutory provision per legal issue.

Load-bearing premise

Step-wise decomposition together with the model's own knowledge for query rewriting will close the wording gap between user questions and statutes enough to raise retrieval accuracy without raising unsupported answers.

What would settle it

Running the same models on a second multi-hop legal benchmark in a different language and finding no rise in retrieval precision or a rise in answers lacking statute support.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Retrieval of relevant statutes becomes more accurate for questions needing several legal provisions.
  • Final answers gain quality because each sub-question links to an explicit statute.
  • Complex reasoning steps become verifiable at the level of individual legal issues.
  • The gains hold for Qwen3-32B and Gemma3-27B on the KoBLEX benchmark.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same split-and-refine pattern could reduce vocabulary mismatch in other retrieval-heavy domains such as medical guidelines.
  • Issue-by-issue statute links could support automated checks for whether an answer stays within cited authorities.
  • Parametric refinement might lower the need for very large external retrieval indexes in statutory tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces Decompose-and-Refine (DaR), a statute-grounded framework for legal question answering that progressively decomposes complex multi-hop questions into atomic sub-questions and generates statute-aligned parametric queries to retrieve the most central statutory provision per legal issue. It is evaluated on the KoBLEX Korean multi-hop LQA benchmark using Qwen3-32B and Gemma3-27B, claiming consistent improvements in retrieval accuracy and final answer quality over existing approaches while enabling transparent issue-level verification of reasoning.

Significance. If the empirical gains hold, the tight integration of decomposition with parametric refinement offers a targeted way to close the vocabulary gap between natural-language questions and statutory text, potentially lowering hallucination risk in multi-hop statutory LQA and improving verifiability. The approach is a direct response to limitations in prior multi-hop QA methods that rely on natural-language reasoning without explicit query reformulation.

major comments (1)
  1. [Abstract] Abstract: the central claim that 'DaR consistently improves both retrieval accuracy and final answer quality' is asserted without any quantitative deltas, baseline names, metric definitions, or error analysis, which is load-bearing for evaluating the empirical contribution.
minor comments (2)
  1. The abstract and introduction would benefit from a one-sentence definition or citation for 'parametric knowledge-based query refinement' on first use.
  2. Consider adding a short illustrative example of a decomposed sub-question and its corresponding parametric query in §3 to clarify the refinement step.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the positive evaluation and the specific suggestion regarding the abstract. The comment identifies a clear opportunity to strengthen the presentation of our empirical results, and we will address it directly in revision.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that 'DaR consistently improves both retrieval accuracy and final answer quality' is asserted without any quantitative deltas, baseline names, metric definitions, or error analysis, which is load-bearing for evaluating the empirical contribution.

    Authors: We agree that the abstract would be strengthened by including concrete quantitative support for the central claim. In the revised manuscript we will update the abstract to report the specific improvements observed (e.g., retrieval accuracy gains and answer-quality metrics) together with the names of the primary baselines, the evaluation metrics used, and a brief indication of the scale of the gains. This change will make the empirical contribution immediately verifiable from the abstract while preserving the existing technical description of the framework. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper proposes the DaR framework as an engineering method for legal QA that integrates decomposition and parametric refinement, then reports empirical gains on the KoBLEX benchmark with Qwen3-32B and Gemma3-27B. No equations, fitted parameters, or predictions appear; the central claim rests on experimental results rather than any self-referential definition or self-citation chain. The derivation is therefore self-contained with independent empirical content.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no information on free parameters, background axioms, or newly postulated entities.

pith-pipeline@v0.9.1-grok · 5794 in / 1004 out tokens · 18348 ms · 2026-06-30T13:20:37.029981+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) have shown strong performance in the legal domain, demonstrating notable potential in Legal Question Answering (LQA). However, unlike general QA, LQA requires answers that are not only accurate but also rigorously grounded in explicit legal authority. In statutory LQA, many questions require multi-hop reasoning across multiple legal issues, substantially increasing the risk of hallucination, thereby making accurate retrieval of supporting statutory provisions a critical prerequisite. Despite recent progress in multi-hop QA, existing approaches often rely on reasoning in natural language or retrieval without explicit query reformulation, leaving the vocabulary gap between user questions and statutory text largely unaddressed. To address this challenge, we propose Decompose-and-Refine (DaR), a statute-grounded LQA framework that tightly integrates step-wise question decomposition with parametric knowledge-based query refinement. DaR progressively decomposes a complex legal question into atomic sub-questions and generates statute-aligned parametric queries for each sub-question, enabling the selection of a single most central statutory provision corresponding to each legal issue. We evaluate DaR on KoBLEX, a Korean multi-hop LQA benchmark grounded in statutory law, using Qwen3-32B and Gemma3-27B. Experimental results demonstrate that DaR consistently improves both retrieval accuracy and final answer quality over existing approaches. Moreover, by explicitly separating sub-questions and their corresponding statutory provisions, DaR facilitates transparent, issue-level verification of complex legal reasoning processes.

Figures

Figures reproduced from arXiv: 2605.24454 by Gary Lee, Hyounghun Kim, Jihyung Lee.

Figure 1
Figure 1. Figure 1: Overview of Decompose-and-Refine ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of Decompose-and-Refine (DAR) framework. Given a legal question, the model iteratively decomposes it into atomic sub-questions. For each sub-question, DAR generates multiple complementary parametric queries, including provision-style queries, sparse keywords, and candidate statute queries. Each query is used for BM25-based lexical retrieval, and the resulting candidate sets are fused via RRF a… view at source ↗
Figure 3
Figure 3. Figure 3: Average number of retrieved statutes per ques [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 55 canonical work pages · 1 internal anchor

  1. [1]

    Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N

    Precise zero-shot dense retrieval without rel- evance labels. InProceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1762–1777, Toronto, Canada. Association for Computational Lin- guistics. Neel Guha, Julian Nyarko, Daniel E. Ho, Christo- pher Ré, Adam Chilton, Aditya Narayana, Alex Chohl...

    work page arXiv 2023

  2. [2]

    Interpretable long-form legal question an- swering with retrieval-augmented large language models. InProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty- Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, AAAI’24/IAAI’24/EAAI...

    work page

  3. [3]

    Gemma 3 Technical Report

    Hallucination-free? assessing the reliability of leading ai legal research tools.Journal of Empiri- cal Legal Studies, 22(2):216–242. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language mod- els. InFindings of the Association for Computational 10 Linguistics: EMN...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  4. [4]

    Read the Background and the CURRENT Question

    work page

  5. [5]

    Decide whether the CURRENT Question is atomic or not

    work page

  6. [6]

    - Rewrite the remaining unresolved part as the Remaining Question

    If NON-ATOMIC: - Extract EXACTLY ONE atomic sub-question that should be solved at this step. - Rewrite the remaining unresolved part as the Remaining Question

    work page

  7. [7]

    Name of Law (Title or Clause Summary) Exact clause text or its key portion

    If ATOMIC: - Treat the CURRENT Question itself as the final Sub-question. - Set Remaining Question to NONE. Guidelines: - Work in Korean when writing questions. - Keep the sub-question as narrow and concrete as 6https://github.com/daehuikim/KoBLEX possible. - The Remaining Question + the solved Sub-question should still be logically equivalent to the orig...

    work page 2020

  8. [8]

    CONSTITUTIONAL COURT ACT / Article. 38 / Time Limit of Adjudication Article 38 (Time Limit of Adjudication) The Constitutional Court shall pronounce the final decision within 180 days after it receives the case for adjudication: Provided, That if the attendance of seven Justices is impossible due to vacancies of Justices, the period of vacancy shall not b...

    work page

  9. [9]

    59 / Effect of Decision Article 59 (Effect of Decision) When a decision ordering the dissolution of a political party is pronounced, the political party shall be dissolved

    CONSTITUTIONAL COURT ACT / Article. 59 / Effect of Decision Article 59 (Effect of Decision) When a decision ordering the dissolution of a political party is pronounced, the political party shall be dissolved. Model Response Sub-Question 1 When the Constitutional Court accepts a case concerning the constitutionality of Party B, within how many days must th...

    work page

  10. [10]

    Constitutional Court Act (Conclusion of Adjudication and Decision) — The Constitutional Court shall pronounce a final decision within 180 days from the date it receives the petition

    work page

  11. [11]

    Constitutional Court Act (Effect of a Decision Ordering Party Dissolution) — A decision ordering the dissolution of a political party results in the party’s assets and facilities being vested in the State and the prohibition of the party’s activities

    work page

  12. [12]

    Constitutional Court unconstitutionality review, political party dissolution, decision period, final decision, adjudication procedure, constitutional complaint

    work page

  13. [13]

    Constitutional Court Act, unconstitutional political party, dissolution adjudication

    work page

  14. [14]

    Constitutional Court Act, time limit for pronouncing a final decision

    work page

  15. [15]

    Constitutional Court Act, political party dissolution procedure Retrieved Context CONSTITUTIONAL COURT ACT / Article. 38 / Time Limit of Adjudication Article 38 (Time Limit of Adjudication) The Constitutional Court shall (...) Sub-Question 2 If a decision ordering the dissolution of Party B is pronounced, what happens to Party B? Parametric Query

    work page

  16. [16]

    Constitutional Court Act (Effect of a Political Party Dissolution Decision) — A decision ordering the dissolution of a political party has the effect of terminating the existence and activities of the party concerned

    work page

  17. [17]

    Constitutional Court Act (Pronouncement of a Political Party Dissolution Decision) — When the Constitutional Court decides to dissolve a political party in a party dissolution adjudication, it shall clearly state the reasons for the decision

    work page

  18. [18]

    Constitutional Court Act (Filing of Petition and Adjudication Procedure) — The Constitutional Court shall conclude oral arguments and pronounce its decision within 180 days from the date it receives the petition for adjudication

    work page

  19. [19]

    Effects of a political party dissolution decision, Constitutional Court, party dissolution procedure, handling of property rights, obligations, disadvantages of a dissolution order, starting point of legal effects

    work page

  20. [20]

    Constitutional Court Act, effect of a political party dissolution decision

    work page

  21. [21]

    Constitutional Court Act, legal consequences following political party dissolution

    work page

  22. [22]

    Political Parties Act, party dissolution decision procedure Retrieved Context CONSTITUTIONAL COURT ACT / Article. 59 / Effect of Decision Article 59 (Effect of Decision) When a decision ordering the dissolution of a (...) Table 7: Successful case study of DAR on a multi-hop statute-based legal question. For brevity, repeated statutory text in the retrieve...

    work page

  23. [23]

    CIVIL ACT / Article. 168 / Causes Interrupting Extinctive Prescription Article 168 (Causes Interrupting Extinctive Prescription) Extinctive prescription shall be interrupted in any of the following cases:1. Demand;2. (...)

    work page

  24. [24]

    279 / Request for Provisional Seizure / Para

    CIVIL EXECUTION ACT / Article. 279 / Request for Provisional Seizure / Para. 1 The following matters shall be entered in a request for provisional seizure:1. Indication of a requested claim, and when such requested claim is not a specific sum of money, the amount converted into the money

    work page

  25. [25]

    Indication of the facts which are to become the reasons for a provisional seizure under Article 277. Model Response Sub-Question 1 Is it possible for Person A to apply for a provisional attachment in order to interrupt the statute of limitations on the debt owed by Person B? Parametric Query

    work page

  26. [26]

    Civil Act (Interruption of Extinctive Prescription) — Extinctive prescription is interrupted upon the occurrence of any of the following grounds: transfer or inheritance of a claim; (...)

    work page

  27. [27]

    Debt, provisional attachment, interruption of extinctive prescription, creditor, debtor, Civil Execution Act

    work page

  28. [28]

    Civil Procedure Act, provisional attachment, interruption of extinctive prescription

    work page

  29. [29]

    Civil Act, claim, extinctive prescription, provisional attachment

    work page

  30. [30]

    Debtor Rehabilitation and Bankruptcy Act, effect of provisional attachment Retrieved Context Civil Act / Article 168(2) (Causes Interrupting Extinctive Prescription) Attachment, provisional attachment, or provisional disposition shall interrupt extinctive prescription. Sub-Question 2 Apart from applying for a provisional attachment, what other measures ca...

    work page

  31. [31]

    Civil Act (Interruption of Extinctive Prescription) — Extinctive prescription shall be interrupted in any of the following cases: demand; attachment, provisional attachment, or provisional disposition; acknowledgment of the obligation by the debtor

    work page

  32. [32]

    Civil Act (Demand) — Where a creditor makes a demand against the debtor, extinctive prescription shall be interrupted; however, if the creditor fails to take further action within six months after becoming aware that the demand is groundless, the prescription shall resume running

    work page

  33. [33]

    Civil Act (Acknowledgment of Obligation) — Where the debtor acknowledges the existence of the obligation or expresses an intention to perform it, extinctive prescription shall be interrupted

    work page

  34. [34]

    Debt, interruption of extinctive prescription, provisional attachment, payment order, lawsuit, creditor, debtor

    work page

  35. [35]

    Civil Procedure Act, interruption of extinctive prescription

    work page

  36. [36]

    Civil Procedure Act, claim, provisional attachment

    work page

  37. [37]

    Civil Act, claim, interruption of extinctive prescription

    work page

  38. [38]

    Table 8: Failure case of DAR caused by error propagation from an incorrect question decomposition to subsequent retrieval steps

    Civil Act, non-performance of obligation Retrieved Context Civil Act / Article 168(1) (Causes Interrupting Extinctive Prescription) A demand shall interrupt extinctive prescription. Table 8: Failure case of DAR caused by error propagation from an incorrect question decomposition to subsequent retrieval steps. 18 Gold Background Person A and Person B enter...

    work page

  39. [39]

    CIVIL ACT / Article. 544 / Delay of Performance and Rescission Article 544 (Delay of Performance and Rescission) If one of the parties does not perform his obligation, the other party may fix a reasonable period and give peremptory notice demanding its performance, and may rescind the contract, if no performance is effected within such period: Provided, T...

    work page

  40. [40]

    CIVIL ACT / Article. 105 / Optional Provisions Article 105 (Optional Provisions) If the parties to a juristic act have declared an intention which differs from any provisions of statutes, which are not concerned with good morals or other social order, such intention shall prevail. Model Response Sub-Question 1 Under what conditions can Person B rescind th...

    work page

  41. [41]

    Civil Act (Rescission of Contract) — Where the obligor delays performance of the obligation, the obligee may fix a reasonable period and urge performance; (...)

    work page

  42. [42]

    Civil Act (Rescission of Contract) — Where there is a special agreement (...)

    work page

  43. [43]

    Sale contract, delay of performance, rescission, conditions for exercising (...)

    work page

  44. [44]

    Civil Act, rescission for delay

    work page

  45. [45]

    Civil Act, non-performance, rescission

    work page

  46. [46]

    Civil Act, delay of performance, right of rescission

    work page

  47. [47]

    Sub-Question 2 How does the situation change if the contract contains other clauses related to rescission? Parametric Query

    Commercial Act, rescission of sale contract Retrieved Context CIVIL ACT / Article 545 (Fixed-Time Acts and Rescission) Article 545 provides that where, due to the nature of the contract or the parties’ declared intentions, the purpose of the contract cannot be achieved unless performance is rendered by a specified date or within a specified period, and on...

    work page

  48. [48]

    Civil Act (Rescission of Contract) — Where the obligor fails to make a genuine (...)

    work page

  49. [49]

    Civil Act (Exercise of the Right of Rescission) — The right of rescission shall (...)

    work page

  50. [50]

    Civil Act (Principle of Priority of Special Agreements) — Where the contracting (...)

    work page

  51. [51]

    Contract rescission, delay of performance, conflict of contractual clauses, (...)

    work page

  52. [52]

    Civil Act, rescission for delay of performance

    work page

  53. [53]

    Civil Act, special agreement clauses on rescission

    work page

  54. [54]

    Civil Act, declaration of intention for rescission

    work page

  55. [55]

    Table 9: Failure case of DAR where correct question decomposition is followed by erroneous statute retrieval

    Civil Act, non-performance, right of rescission Retrieved Context ACT ON THE REGULATION OF TERMS AND CONDITIONS / Article 5 Article 5 governs the interpretation of standardized contract terms and provides guiding principles for resolving ambiguities or conflicts in contractual clauses. Table 9: Failure case of DAR where correct question decomposition is f...

    work page