Pith. sign in

REVIEW 3 major objections 5 minor 33 references

LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LegalReasoner claims that legal judgment prediction is improved by supervising each reasoning step, not just the final verdict.

desk verdict A serious engineering contribution undermined by a likely label-leakage flaw in the dataset construction; the method and dataset deserve a chance, but the central empirical claim as stated is not supported. read the letter →

arxiv 2506.07443 v1 pith:C36BNTDI submitted 2025-06-09 cs.AI

Weijie Shi , Han Zhu , Jiaming Ji , Mengze Li , Jipeng Zhang , Ruiyuan Zhang , Jia Zhu , Jiajie Xu
show 2 more authors
Sirui Han Yike Guo
This is my paper
classification cs.AI
keywords legaljudgmentpredictionprocessverificationstep-wisereasoningerrorcorrectiondisputeidentificationHKlargelanguagemodelsHongKongcourtcases
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that legal judgment prediction fails mainly because errors accumulate inside long reasoning chains, not because the model lacks legal knowledge. LegalReasoner first decomposes a case into dispute points, then produces a step-by-step legal analysis while a process verifier scores every step on correctness, progressiveness, and potential. Flagged steps are attributed to one of eight expert-defined error types and repaired with tailored strategies such as statute retrieval or fact tracking. On the released LegalHK benchmark of 58,130 Hong Kong cases, the authors report that this raises concordance with court decisions on an open-source 70B model from 72.37 to 80.27, surpassing GPT-o1 on both LegalHK and CMDL.

What carries the argument

The load-bearing mechanism is the fine-tuned process verifier. For each reasoning step, three signals are computed: Correctness, which checks alignment with case facts and legal logic; Progressiveness, which checks that the step advances toward a judgment; and Potential, the fraction of sampled continuations that terminate in the court's actual decision. The verifier is trained on the minimum of these three scores, deliberately avoiding averages that could hide one failing aspect; a step whose score falls below threshold T is sent to a classifier that assigns one of eight error types, each tied to a correction strategy such as statute retrieval, fact tracking, compensation validation, or enforcement analysis. The Reasoner itself is fine-tuned on LegalHK's dispute points and step-wise reasoning chains, so the whole system couples case decomposition with per-step audit.

What would settle it

Re-run LegalReasoner on LegalHK test cases using only the original fact portions that existed before the GPT-4 enhancement step, removing every sentence traceable to the court-reasoning or judgment fields; if case-level accuracy falls from 80.27 toward the 72.37 fine-tuning baseline, the verification-correction gain is an artifact of outcome leakage.

Watch

Extended reading notes

Core claim

The central claim is that supervising the reasoning process itself, rather than only the final ruling, is what makes legal judgment prediction reliable. LegalReasoner treats judicial analysis as a sequence of steps, scores each step with a trained verifier that takes the minimum of correctness, progressiveness, and potential, and, when that minimum falls below a threshold, classifies the error and activates an expert-designed correction path. The paper reports that this pipeline improves case-level accuracy from 72.37 to 80.27 on LegalHK with an open-source 70B model and outperforms GPT-o1 on both LegalHK and CMDL. Ablations show that removing step-wise reasoning is the largest single loss, and that verification-correction contributes more than dispute identification alone.

Load-bearing premise

The comparisons stand on the assumption that the GPT-4-augmented fact descriptions contain enough information to reproduce the court's decision without revealing it; if outcome-derived wording leaked into those descriptions, the reported gain is prediction from the label rather than from case facts.

Editorial extensions

If this is right

  • Concordance with court decisions rises from 72.37 to 80.27 on LegalHK when verification-correction is added to the fine-tuned 70B reasoner.
  • Removing step-wise reasoning is the most damaging ablation, dropping case-level accuracy by 10.78 on LegalHK, while removing dispute identification costs another 4.32.
  • Verification-correction beats Best-of-N sampling: the best Best-of-N variant reaches 76.82 CL-Acc on LegalHK with ten samples, while LegalReasoner reaches 80.27 on a single path.
  • The same pipeline transfers to the Chinese criminal CMDL dataset, where LegalReasoner reaches 76.95 CL-Acc, above GPT-o1's 72.03.
  • The released 58,130-case LegalHK dataset with dispute points, reasoning chains, and verification labels is a resource for training and evaluating similar step-wise legal reasoners.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The eight-way error taxonomy is itself a transferable asset: it gives future systems a vocabulary for explaining why a legal step failed, which matters for auditability independent of accuracy gains.
  • A decisive check the paper does not report is leakage sensitivity, since fact descriptions were augmented from court reasoning and judgments; re-running on the un-augmented original facts would reveal how much of the 7.9-point gain is genuine reasoning rather than label-derived wording.
  • If the verifier generalizes, the same min-of-three scoring could be applied to other document-grounded reasoning tasks where intermediate traces and gold outcomes are available, such as medical or regulatory decision support.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. LegalReasoner proposes a two-stage pipeline for legal judgment prediction (LJP): a reasoner that identifies dispute points and produces step-wise reasoning, and a process verifier that scores each reasoning step on correctness, progressiveness, and potential, triggering expert-designed correction strategies when errors are detected. The paper introduces LegalHK, a dataset of 58,130 Hong Kong court cases annotated with dispute points, reasoning chains, and process verification labels. Experiments on LegalHK and CMDL report case-level concordance of 80.27% and 76.95% (CL-Acc), surpassing GPT-o1 and improving over fine-tuned LLAMA-3.1-70B by 7.90 and 8.74 points respectively. The central claim is that process-level verification and correction significantly improve agreement with court decisions.

Significance. If the results are valid, this would be a notable contribution: it is, to my knowledge, the first LJP method that applies step-wise process verification with targeted correction and releases a large annotated dataset for training and evaluation. The ablation design (Table 2, Figure 3) is thoughtful, and the open release of LegalHK is valuable. However, the main empirical claim rests on the integrity of the fact descriptions in LegalHK. Because the dataset-construction pipeline explicitly augments facts using the court's own reasoning and final judgment, the reported concordance may substantially reflect label leakage rather than prediction from original case facts. The paper's significance therefore hinges on a data-construction issue that the current manuscript does not resolve.

major comments (3)
  1. [§4.1, Appendix A.2, Table 3] The fact descriptions used as model inputs are constructed with access to the gold outcome. Step 2 of the Refinement and Filtering in §4.1 states that the authors 'augmented it using GPT-4 based on the original document, extracted court reasoning, and legal judgment,' and the Fact Enhancement prompt in Table 3 explicitly instructs GPT-4 to review the court's reasoning section and to populate a JSON field 'support&reject' with the known decision. The verification procedure described immediately afterward only screens for explicit revelation of the judgment, not for subtler outcome signals such as selective inclusion of expert valuations, credit judgments, or framing. The example in Figure 5 illustrates the risk: the fact description includes the court-reasoning-derived statement that 'Experts Mr. Zorn and Mr. Peyer valued the quota rights at between $50,000 to $55,000,' which strongly suggests the conclusion that the $5,500 sale was an undervalue. Consequently, the central result of the paper—80.27% CL-Acc on LegalHK, and by extension the comparisons against GPT-o1 and the ablations—does not establish prediction from case claims and unaltered facts; it may partly measure the model's ability to exploit outcome-derived content embedded in the input.
  2. [§4.2.1, §5.5] The process verifier's training and evaluation are partly circular. In Eq. (2), the Potential score uses the true court decision d* to label sampled completions, which is a legitimate form of outcome supervision for a verifier. However, the Correctness and Progressiveness scores are generated by GPT-4 comparisons between 'case facts, annotated court reasoning, and judgment decisions'; since the 'case facts' in LegalHK were themselves enhanced using the court reasoning and judgment, the verifier is trained to reward steps that align with the gold outcome rather than with logically sound reasoning from neutral facts. The annotation-quality evaluation in §5.5 is also measured against automatically generated labels, with only 160 manually annotated steps; the reported 85.7% verification accuracy therefore does not break the circularity. This issue is load-bearing because the paper's main contribution is the verification-correction mechanism, and the reported gains of that mechanism (Table 2) may be inflated by the same leakage.
  3. [§5.1.1, §5.2] The manuscript provides no evaluation on unenhanced fact descriptions, even though the original court documents are the starting point of the pipeline. The same refinement and filtering process described in §4.1 is applied to CMDL (§5.1.1), so the leakage concern is not limited to LegalHK. A minimal control experiment would be to run LegalReasoner and the strongest baselines on the original, unmodified facts (or on facts augmented without access to the court reasoning and judgment) and report CL-Acc. Without such a control, the abstract's claim of improving concordance from 72.37 to 80.27 cannot be separated from the effect of outcome-aware input construction, and the same holds for the per-component ablations. This missing control is not a cosmetic issue; it is essential for interpreting every quantitative result in Tables 1 and 2.
minor comments (5)
  1. [§4.2.1] Equation (1) is labeled '(2)' in the manuscript; the equation numbers should be corrected to match the order of appearance.
  2. [Appendix C] The text refers to 'CA-Acc' in the sentence describing Figure 7; this should be 'CL-Acc' for consistency with the rest of the paper.
  3. [Figure 2] In the verifier pipeline, the number 0.33 appears as a component score; the text states the final score is the minimum of the three scores, but the figure could clarify whether 0.33 is an example minimum or a threshold value. The caption should be updated accordingly.
  4. [Limitations] The Limitations section discusses computational cost, annotation noise, and generalizability, but it does not acknowledge the possibility of outcome leakage in the fact-enhancement step. I recommend adding a discussion of this risk and the steps taken to mitigate it, or a description of a leakage test if one was performed.
  5. [Data release] The data is available at the stated HuggingFace URL, but no information is given about code release; for reproducibility, please include the code for dataset construction and model training, or state explicitly that it will not be released.

Circularity Check

2 steps flagged · score 7.0 of 10

LegalHK (and CMDL, per Sec. 5.1.1) fact descriptions are augmented using the court's own reasoning and final judgment, so the reported 80.27 concordance is partly a re-reading of label-derived inputs rather than prediction from original case facts.

  1. self definitional [Section 4.1 (Refinement and Filtering), Appendix A.2 (Fact Description Enhancement and Verification), Table 3]
    "To ensure the fact description contained sufficient knowledge to produce judgments consistent with the court’s decisions, we augmented it using GPT-4 based on the original document, extracted court reasoning, and legal judgment. ... [Table 3] Please review the court’s reasoning section and extract any additional factual information to supplement the existing facts ... Requirements: 1. Review court_reasoning to identify: - Facts mentioned in decision explanation - Facts referenced in evidence analysis - Facts cited when applying legal principles - Facts supporting final judgment ..."

    The model's input fact description is not the original case fact record; it is constructed by GPT-4 from the court's reasoning and final judgment, with the prompt explicitly instructing extraction of facts supporting the final judgment and requiring a support/reject outcome field. The target decision d* is therefore used to select and shape the features from which d* is predicted, so the reported concordance partially measures the model's ability to exploit label-derived descriptions. The subsequent GPT-4 and manual verification only checks for explicit verdict leakage and does not remove this outcome-based selection bias.

  2. self definitional [Section 5.1.1 (Datasets)]
    "In addition to our constructed LegalHK dataset, we utilized the latest CMDL dataset (Huang et al., 2024), both of which underwent refinement and filtering processes as described in Section 4.1."

    CMDL is presented as an external validation set, but the paper states it underwent the same Section 4.1 pipeline, which includes the target-derived fact-enhancement step. Hence the CMDL results, including the claimed advantage over GPT-o1, are subject to the same label-leakage concern and do not provide an independent, leakage-free check of the method.

full rationale

The paper's key evaluation claim is that LegalReasoner improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B using case claims and fact descriptions as input. However, the LegalHK fact descriptions were not taken as-is from the original cases; they were augmented by GPT-4 using the court's reasoning and legal judgment, with a prompt that directs extraction of facts supporting the final judgment. This makes the input feature set partially defined in terms of the target outcome, so the measured concordance is not a clean prediction from original case facts. The same pipeline is applied to the CMDL benchmark, removing it as an independent external validation. The paper's own verification step only filters explicit verdict sentences and does not remove subtler outcome signals. I am not flagging the process verifier's use of the true court decision in training labels as circular, because supervised training on gold labels is standard; the circularity is specifically in the construction of the test-time input. The method's internal ablations still show that verification-correction contributes, so this is not a case where the entire method is vacuous, but the headline benchmark results are substantially compromised by construction.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The empirical claims rest on the LegalHK construction pipeline and on GPT-4 based annotation and evaluation. The dominant load-bearing choices are the enhancement of fact descriptions with court reasoning and judgment, the binary support/reject framing, and the process-verifier label definitions.

free parameters (3)
  • Verification threshold T = 0.5
    Hand-set threshold for flagging a reasoning step as erroneous. No sensitivity analysis is reported, and the behavior of the entire verification-correction loop depends on it.
  • Number of potential samples N = not specified
    The Potential score samples N completions to estimate whether the current reasoning sequence can reach the true court decision; N is not reported.
  • Document compression ratio = one-third to one-half of original length
    LLM-based compression is applied before extraction; the compression level is a design choice that could affect which facts and reasoning survive.
assumptions (6)
  • domain assumption Hong Kong court decisions are reducible to a binary support/reject label plus listed judgment elements.
    The task definition in Section 3 encodes d in {0,1}, and the evaluation compares predicted elements with court rulings. This assumes a lossy representation captures the legal outcome.
  • domain assumption GPT-4 extracts and annotates structured legal information with near-human reliability.
    Dataset construction, fact enhancement, verification labels, and element metrics all rely on GPT-4 outputs. Only a 160-step manual sample is used to validate verifier labels.
  • domain assumption Fact descriptions augmented from court reasoning do not prematurely reveal judgment outcomes.
    Appendix A.2 states a two-tier verification checked for leakage, but the enrichment source is the court's own reasoning and judgment, so the assumption is load-bearing and questionable.
  • ad hoc to paper The eight expert-designed error types cover all important legal reasoning errors.
    Section 4.2.2 and Appendix B introduce eight error categories and corresponding fixes. If a real error type is missing, the correction module cannot address it.
  • domain assumption GPT-4 based element coverage and precision evaluation matches human evaluation.
    Section 5.1.2 states manual verification confirms alignment, but the full-scale metric is computed by GPT-4 without reporting the manual agreement rate.
  • domain assumption The Potential score, which samples completions toward the true court decision d* , is a valid training signal for the verifier.
    This is a standard process-reward construction, but it embeds the label into the verifier's training objective and assumes the sampled completions are representative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning." pith.science (2026). https://pith.science/paper/C36BNTDI

@misc{pith2026250607443,
  author       = {Pith},
  title        = {Pith review of: LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C36BNTDI}},
  note         = {Machine review of arXiv:2506.07443}
}
read the original abstract

Legal judgment prediction (LJP) aims to function as a judge by making final rulings based on case claims and facts, which plays a vital role in the judicial domain for supporting court decision-making and improving judicial efficiency. However, existing methods often struggle with logical errors when conducting complex legal reasoning. We propose LegalReasoner, which enhances LJP reliability through step-wise verification and correction of the reasoning process. Specifically, it first identifies dispute points to decompose complex cases, and then conducts step-wise reasoning while employing a process verifier to validate each step's logic from correctness, progressiveness, and potential perspectives. When errors are detected, expert-designed attribution and resolution strategies are applied for correction. To fine-tune LegalReasoner, we release the LegalHK dataset, containing 58,130 Hong Kong court cases with detailed annotations of dispute points, step-by-step reasoning chains, and process verification labels. Experiments demonstrate that LegalReasoner significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. The data is available at https://huggingface.co/datasets/weijiezz/LegalHK.

Figures

Figures reproduced from arXiv: 2506.07443 by the authors.

Figure 1
Figure 1. An illustration of legal judgment prediction. LegalReasoner first breaks down the process into identifying [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Automatic annotation for step-wised verification-correction. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of Best of N (BON) and our approach, where SC is Self-consistency, OV is Outcome Verifier, GPV is General Process Verifier, and LPC is Legal Process Verifier. Best of N approaches with 76.82 on LegalHK and 74.55 on CMDL, while general approaches achieve around 74-75 on LegalHK and 71-72 on CMDL. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Illustration of LegalHK: an annotated legal judgment dataset with disputes and reasoning. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Overview of error types and corresponding correction strategies in LegalReasoner. The framework consists [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Impact of different correction strategies on [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 29 canonical work pages

  1. [1]

    Adel Ammar, Anis Koubaa, Bilel Benjdira, Omer Nacar, and Serry Sibaee. 2024. Prediction of arabic legal rulings using large language models. Electronics, 13(4):764

    work page 2024

  2. [2]

    Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in english. arXiv

    work page 2019

  3. [3]

    Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. Legal-bert: The muppets straight out of law school. arXiv

    work page 2020

  4. [4]

    Jiaxi Cui, Munan Ning, Zongjian Li, Bohua Chen, Yang Yan, Hao Li, Bin Ling, Yonghong Tian, and Li Yuan. 2024. Chatlaw: A multi-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model. arXiv

    work page 2024

  5. [5]

    Junyun Cui, Xiaoyu Shen, and Shaochun Wen. 2023. A survey on legal judgment prediction: Datasets, metrics, models and challenges. IEEE Access

    work page 2023

  6. [6]

    Francisco de Arriba-P \'e rez, Silvia Garc \' a-M \'e ndez, Francisco J Gonz \'a lez-Casta \ n o, and Jaime Gonz \'a lez-Gonz \'a lez. 2022. Explainable machine learning multi-label classification of spanish legal judgements. Journal of King Saud University-Computer and Information Sciences, 34(10):10180--10192

    work page 2022

  7. [7]

    Aniket Deroy, Kripabandhu Ghosh, and Saptarshi Ghosh. 2024. Ensemble methods for improving extractive summarization of legal case judgements. Artificial Intelligence and Law, 32(1):231--289

    work page 2024

  8. [8]

    Yi Feng, Chuanyi Li, and Vincent Ng. 2022 a . Legal judgment prediction: A survey of the state of the art. In IJCAI, pages 5461--5469

    work page 2022

Show all 33 references
  1. [9]

    Yi Feng, Chuanyi Li, and Vincent Ng. 2022 b . Legal judgment prediction via event extraction with constraints. In ACL, pages 648--664

    2022

  2. [10]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv

    2023

  3. [11]

    Wanhong Huang, Yi Feng, Chuanyi Li, Honghan Wu, Jidong Ge, and Vincent Ng. 2024. Cmdl: A large-scale chinese multi-defendant legal judgment prediction dataset. In Findings of ACL, pages 5895--5906

    2024

  4. [12]

    Yunyun Huang, Xiaoyu Shen, Chuanyi Li, Jidong Ge, and Bin Luo. 2021. Dependency learning for legal judgment prediction with a unified text-to-text transformer. arXiv

    2021

  5. [13]

    Wonseok Hwang, Dongjun Lee, Kyoungyeon Cho, Hanuhl Lee, and Minjoon Seo. 2022. A multi-task benchmark for korean legal language understanding and judgement prediction. NIPS, 35:32537--32551

    2022

  6. [14]

    Maor Ivgi, Uri Shaham, and Jonathan Berant. 2023. Efficient long-text understanding with short-text models. TACL, 11:284--299

    2023

  7. [15]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv

    2023

  8. [16]

    Dugang Liu, Weihao Du, Lei Li, Weike Pan, and Zhong Ming. 2022. Augmenting legal judgment prediction with contrastive case relations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2658--2667

    2022

  9. [17]

    Yifei Liu, Yiquan Wu, Yating Zhang, Changlong Sun, Weiming Lu, Fei Wu, and Kun Kuang. 2023. Ml-ljp: multi-law aware legal judgment prediction. In SIGIR, pages 1023--1034

    2023

  10. [18]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. NIPS, 36

    2024

  11. [19]

    Mihai Masala, Traian Rebedea, and Horia Velicu. 2024. Improving legal judgement prediction in romanian with long text encoders. arXiv

    2024

  12. [20]

    Cristian Millan-Arias, Marie Chantelle C Medina, Jean Felipe C Ferreira, Lucas M Da Silva Oliveira, Allana Rocha, Leandro Honorato de S Silva, Cleyton Mario O Rodrigues, Jo \ a o Fausto L de Oliveira, Paulo Christiano Sobral, Nathalia Cau \'a s, et al. 2023. Legal judgement pr...

    2023

  13. [21]

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor R \"u hle, Yuqing Yang, Chin-Yew Lin, et al. 2024. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. arXiv

    2024

  14. [22]

    Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2023. Refiner: Reasoning feedback on intermediate representations. arXiv

    2023

  15. [23]

    TYSS Santosh, Mohamed Hesham Elganayni, Stanis aw S \'o jka, and Matthias Grabmair. 2024. Incorporating precedents for legal judgement prediction on european court of human rights cases. arXiv

    2024

  16. [24]

    Jingyun Sun, Shaobin Huang, and Chi Wei. 2024. Chinese legal judgment prediction via knowledgeable prompt learning. Expert Systems with Applications, 238:122177

    2024

  17. [25]

    Dietrich Trautmann, Alina Petrova, and Frank Schilder. 2022. Legal prompt engineering for multilingual legal judgement prediction. arXiv

    2022

  18. [26]

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistic...

    2024

  19. [27]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv

    2022

  20. [28]

    Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. 2022. Re3: Generating longer stories with recursive reprompting and revision. arXiv

    2022

  21. [29]

    Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. Rest-mcts*: Llm self-training via process reward guided tree search. arXiv

    2024

  22. [30]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. NIPS, 36:46595--46623

    2023

  23. [31]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv

    2024

  24. [32]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  25. [33]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.