Pith. sign in

REVIEW 5 major objections 6 minor 55 references

Can Large Language Models Predict the Outcome of Judicial Decisions?

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 3-billion-parameter LLaMA model fine-tuned on a new Arabic court dataset predicts legal judgments within a few points of an 8-billion-parameter model, at a fraction of the resource cost.

desk verdict A useful Arabic legal dataset is buried under a task-design flaw: the prompt asks models to 'extract' the judgment, so the benchmark likely measures copying, not prediction. read the letter →

arxiv 2501.09768 v3 pith:NZF3GSJJ submitted 2025-01-15 cs.CL cs.AI

Mohamed Bayan Kmainasi , Ali Ezzat Shahroor , Amani Al-Ghraibah This is my paper · ORCID
classification cs.CLcs.AI
keywords legaljudgmentpredictionArabicNLPlargelanguagemodelsLoRAfine-tuninglow-resourcelanguagesinstructionfollowingSaudicommercialcourtBERTScore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that legal judgment prediction (LJP) in Arabic, a low-resource language, is feasible with open-source language models, provided they are fine-tuned efficiently. The authors built a dataset from Saudi commercial court judgments, split each case into 'Facts + Reasons' and final 'Judgment' text, and crafted 75 Arabic instructions to fine-tune LLaMA-3.2-3B and LLaMA-3.1-8B with LoRA. Across BLEU, ROUGE, and BERTScore, the fine-tuned 3B model lands close to the 8B model (BERTScore 0.74 vs 0.76), and the authors argue that this makes smaller models a resource-efficient option. The paper also reports statistically significant gains from instruction diversity and releases the dataset, code, and models.

What carries the argument

The mechanism that carries the argument is LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning method that updates only small low-rank matrices instead of all model weights, combined with a custom Arabic instruction-following dataset. The task is formalized as $f(\text{Instruction}, \text{Facts} + \text{Reasons}) \rightarrow \text{Judgment}$, and one-shot examples are retrieved with a Sentence-BERT multilingual embedding model. LoRA allows the 3B model to be fine-tuned on a single GPU and to reach performance close to the 8B model, while the instruction-diverse training set is credited with improving generalization across the 75 prompt styles.

What would settle it

Measure the ROUGE-L score between the 'Reasons' and 'Judgment' parts of the released dataset; if the score is high, say above 0.7, the task reduces to extraction rather than prediction.

Watch

Extended reading notes

Core claim

The central claim is that LoRA fine-tuning on a modest Arabic instruction-following dataset narrows the gap between a 3B and an 8B open-source LLM to a few points on legal judgment prediction. On the authors' test set, the fine-tuned LLaMA-3.2-3B reaches BERTScore 0.74, BLEU 0.24, ROUGE-1 0.50, and ROUGE-2 0.39, while the fine-tuned LLaMA-3.1-8B reaches 0.76, 0.26, 0.53, and 0.41, respectively; both far exceed their zero-shot baselines (BERTScore 0.54 and 0.58). The paper interprets this as evidence that efficient parameter adaptation, not raw model size, is the main driver of LJP performance in this setting, and that a diverse set of 75 Arabic instructions yields statistically significant improvements in instruction generalization. It also introduces what it calls the first Arabic instruction-following dataset for LJP, constructed from public Saudi commercial court judgments and made publicly available.

Load-bearing premise

The load-bearing premise is that the 'Judgment' text is not already contained in the 'Facts + Reasons' input; if it is, the models are being evaluated on extraction and copying rather than on predicting judicial outcomes.

Editorial extensions

If this is right

  • Fine-tuned 3B models become a practical choice for Arabic legal NLP on modest hardware, since the BERTScore gap to the 8B model is only 0.02.
  • LoRA fine-tuning beats one-shot in-context learning on every reported metric, suggesting parameter-efficient adaptation is the better deployment strategy for legal documents.
  • Instruction diversity during fine-tuning produces statistically significant gains, indicating that prompt variation improves a model's ability to follow unseen Arabic legal instructions.
  • The released dataset, code, and models give other researchers a starting point for Arabic legal NLP and for adapting the pipeline to other low-resource languages.
  • Because the difference between the 3B and 8B fine-tuned models is under 3% on average, model size is not the dominant factor after fine-tuning on a domain dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the final judgment text is largely recoverable from the 'Reasons' portion of the input, the reported numbers may measure extractive copying rather than genuine outcome prediction; a Facts-only evaluation would settle this.
  • A stronger test would recast LJP as classification over a fixed set of outcome classes and measure accuracy on cases where the judgment is truly future information, rather than text overlap.
  • The LLM-based qualitative scores were produced by the same family of models being evaluated, so the 'Legal Language' and 'Clarity' judgments may partly reflect stylistic similarity to the reference; a human-annotated subset would test this.
  • The resource-efficiency claim could be quantified in terms of training time, GPU memory, and cost per prediction, allowing practitioners to weigh a 2-3% score gap against compute savings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces a new Arabic Legal Judgment Prediction (LJP) dataset constructed from Saudi commercial court judgments, and benchmarks two open-source LLMs (LLaMA-3.2-3B-Instruct and LLaMA-3.1-8B-Instruct) under zero-shot, one-shot, and LoRA fine-tuning settings. The authors evaluate performance using BLEU, ROUGE, BERTScore, and an LLM-based qualitative scoring, and report that fine-tuned smaller models achieve comparable performance to larger models. They also analyze instruction generalization after fine-tuning on 75 Arabic instructions, and release the dataset, code, and models.

Significance. If the benchmark is valid, the paper makes a useful contribution to Arabic legal NLP: it provides a new domain-specific dataset, releases code and models, and offers a practical comparison of parameter-efficient fine-tuning for a low-resource language. The resource-efficiency claim (fine-tuned 3B model close to 8B model) is practically relevant. However, two load-bearing issues undermine the current interpretation: (1) the task may be extraction rather than prediction because the input includes 'Reasons' and the prompt explicitly asks to 'extract the final judgment text', and (2) the instruction-generalization evaluation uses the same 75 instructions that were used to build the fine-tuning data. Both points need to be resolved before the central claims can be accepted.

major comments (5)
  1. [Section III.D.1 and Section III.A] The task formalization f(Instruction, Facts + Reasons) -> Judgment uses the example instruction 'analyze the reasons and extract the final judgment text.' Because the dataset is created by parsing court judgments into 'Facts + Reasons' and 'Judgment' components, the gold judgment may be largely recoverable from the 'Reasons' portion. The paper provides no overlap statistics (e.g., ROUGE or BERTScore between the input's Reasons and the gold Judgment), no manual audit of test instances, and no ablation that removes the Reasons component. Without such evidence, the high BLEU/ROUGE/BERTScore values in Table I may reflect copying and rewriting rather than legal outcome prediction, which would invalidate the paper's central claim. Please add quantitative overlap analysis, show representative test examples with the corresponding gold judgments, and run an ablation that uses only the Facts (or a summarized version) as input.
  2. [Section V.A and Section III.A] The instruction-generalization analysis evaluates models on the same 75 instructions that were used to construct the fine-tuning data in Section III.A. The reported improvement in BERT F1 after fine-tuning is therefore expected from memorization of the exact instruction prompts and does not demonstrate generalization to new instructions. The paper must evaluate on held-out instructions (e.g., a subset reserved from the 75, or a fresh set of instructions written specifically for evaluation) to support the 'generalization' and 'human-centric' claims.
  3. [Section III.B] The fine-tuning description is internally inconsistent: it states 'Both models were trained for two epochs on a single A100 GPU' and later 'Training was performed for one epoch using the AdamW optimizer.' The number of training epochs must be clarified, since the resource-efficiency and convergence claims depend on it, and reproducibility requires an unambiguous protocol.
  4. [Section IV and Table I] The main quantitative claim of 'comparable performance' between LLaMA-3.2-3B-FT and LLaMA-3.1-8B-FT rests on point estimates (e.g., BERTScore 0.74 vs. 0.76) without confidence intervals or paired significance tests on the 538 test samples. Given the small test set and the close margins, the paper should report bootstrap confidence intervals or a paired test (e.g., Wilcoxon signed-rank on per-sample BLEU, ROUGE, and BERTScore values) to support the comparability claim.
  5. [Section IV.A] The qualitative evaluation uses LLaMA-3.1-8B-Instruct as a judge without any validation against human ratings. The eight-dimensional scores are presented as if they are reliable measures of response quality, but the paper does not report agreement with human annotators (e.g., correlation or percentage agreement on a sample). Please provide such validation or explicitly discuss the limitation and justify the use of the LLM judge for Arabic legal text.
minor comments (6)
  1. [Section V.B] There is a typo: 'finetning' should be 'fine-tuning'; additionally, 'LLama' appears in Section I and the caption of Figure 1, and the metric name 'BERT' in Table I should be 'BERTScore' to match the text.
  2. [Section V.A] The reported 'p-value of 0' is not a valid p-value; it should be reported as p < 0.001 or with the actual computed value.
  3. [Section III.B and Section III.E] The training setup mentions a single A100 GPU while inference is described using four NVIDIA A16 GPUs; clarify which hardware was used for each stage and why the configuration differs.
  4. [Section III.A] Figure 2 is described as 'a sample data point' but not explained in the text; include a brief description of the fields and the structure of the dataset sample.
  5. [References] Reference [17] is a self-citation to the authors' prior work on native vs. non-native prompting; ensure the comparison is presented with appropriate context and not overgeneralized.
  6. [Dataset release] The paper should include a statement on data licensing, privacy, and ethical use of the scraped court judgments, including any restrictions on redistribution of the dataset.

Circularity Check

2 steps flagged · score 6.0 of 10

The instruction-generalization claim is evaluated on the same 75 instructions used for fine-tuning, and the LJP task is defined as extracting the final judgment text from the Reasons; both reduce reported improvements to fitting or copying rather than prediction.

  1. fitted input called prediction [Section V.A (Instruction Generalization Analysis); instructions built in Section III.A]
    "The models were trained on a set of 75 diverse instructions to encourage generalization and promote human-centric behavior. ... the 75 instructions were distributed across the test set, and for each instruction, we computed the BERT F1 score."

    Section III.A states: 'We created 75 diverse Arabic instructions using GPT-4o varying in length and complexity. These instructions were uniformly distributed across the dataset’s data points to ensure broad coverage.' These same 75 instructions were used to build the fine-tuning examples. Section V.A then measures 'generalization' by computing BERT F1 on those same 75 instructions in the test set and running a Wilcoxon test on the improvement. The post-fine-tuning gains (0.54 to 0.64 and 0.58 to 0.74) are therefore evaluations on the training distribution: the model has been fine-tuned on the exact instruction set whose generalization is being measured.

  2. self definitional [Section III.D.1 (Zero-shot Prompting), contrasted with Introduction's LJP definition]
    "f (Instruction, Facts + Reasons ) → Judgment ... Instruction: A prompt specifying the task, e.g., ”Based on the facts, analyze the reasons and extract the final judgment text. ”"

    The Introduction defines LJP as 'predicts judicial outcomes based on factual case details,' but the operational task in Section III.D.1 feeds the model Facts+Reasons and instructs it to 'extract the final judgment text' from the Reasons. This defines the target output as a component of the input: the model is scored on how well it copies or rewrites a judgment that the prompt assumes is contained in the Reasons. The paper reports no overlap statistics between gold judgments and Reasons and no ablation removing Reasons, so the high BLEU/ROUGE/BERTScore values in Table I cannot be distinguished from extraction fidelity. The 'prediction' of judicial outcomes is thus a renaming of extraction by construction.

full rationale

The two flagged steps are the load-bearing reductions. First, the instruction-generalization analysis in Section V.A is circular because the 75 instructions used to evaluate generalization are exactly the 75 instructions on which the models were fine-tuned (Section III.A); the measured improvement is a training-distribution effect, not evidence of generalization to unseen instructions. Second, the central LJP claim is self-definitional: the task is formalized as f(Instruction, Facts+Reasons) → Judgment, with the example instruction explicitly asking the model to 'extract the final judgment text' from the Reasons, so the output is defined as recoverable from the input. Without overlap statistics or a Reasons-ablation, the reported metrics measure copying/rewriting rather than legal outcome prediction. I did not count the use of LLaMA-3.1-8B-Instruct as a qualitative judge as circular, because it is a separate checkpoint from the fine-tuned models; that is a methodological bias concern, not a definitional reduction. No load-bearing self-citation chain is present: the authors' prior works (references [17] and [36]) appear only as related work and do not justify the core result. Overall, partial circularity is present in the instruction-generalization claim and in the task definition itself, but the raw fine-tuning-versus-base-model comparison in Table I still has independent empirical content; hence score 6 rather than 8 or 10.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper relies on several domain assumptions that are not empirically tested: the non-overlap of judgment and facts/reasons, the validity of n-gram metrics for legal text, the reliability of LLM-based scoring, and the representativeness of the sampled dataset. No new entities are postulated.

assumptions (4)
  • ad hoc to paper The judgment text is not trivially derivable from the Facts + Reasons input; the task is prediction, not extraction.
    Section III.D.1 prompts the model to 'extract the final judgment text', implying the judgment appears in the source document. If the judgment text is already present in the Facts/Reasons, the evaluation measures copying, not prediction.
  • domain assumption BLEU and ROUGE are appropriate evaluation metrics for legal judgment generation.
    Section III.C uses BLEU and ROUGE, which are n-gram overlap metrics that may not capture the semantic validity of a legal judgment, especially for low-resource Arabic.
  • domain assumption LLM-based scoring with LLaMA-3.1-8B-Instruct provides a reliable proxy for human qualitative assessment.
    Section III.C uses LLaMA-3.1-8B as the judge, but no validation against human annotations is provided.
  • domain assumption The dataset split (3752 train / 538 test) is representative of the full Saudi commercial court distribution.
    Section III.A states a smaller version was sampled, but the sampling strategy is not described, leaving open the possibility of selection bias.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can Large Language Models Predict the Outcome of Judicial Decisions?." pith.science (2026). https://pith.science/paper/NZF3GSJJ

@misc{pith2026250109768,
  author       = {Pith},
  title        = {Pith review of: Can Large Language Models Predict the Outcome of Judicial Decisions?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NZF3GSJJ}},
  note         = {Machine review of arXiv:2501.09768}
}
read the original abstract

Large Language Models (LLMs) have shown exceptional capabilities in Natural Language Processing (NLP) across diverse domains. However, their application in specialized tasks such as Legal Judgment Prediction (LJP) for low-resource languages like Arabic remains underexplored. In this work, we address this gap by developing an Arabic LJP dataset, collected and preprocessed from Saudi commercial court judgments. We benchmark state-of-the-art open-source LLMs, including LLaMA-3.2-3B and LLaMA-3.1-8B, under varying configurations such as zero-shot, one-shot, and fine-tuning using LoRA. Additionally, we employed a comprehensive evaluation framework that integrates both quantitative metrics (such as BLEU, ROUGE, and BERT) and qualitative assessments (including Coherence, Legal Language, Clarity, etc.) using an LLM. Our results demonstrate that fine-tuned smaller models achieve comparable performance to larger models in task-specific contexts while offering significant resource efficiency. Furthermore, we investigate the impact of fine-tuning the model on a diverse set of instructions, offering valuable insights into the development of a more human-centric and adaptable LLM. We have made the dataset, code, and models publicly available to provide a solid foundation for future research in Arabic legal NLP.

Figures

Figures reproduced from arXiv: 2501.09768 by the authors.

Figure 1
Figure 1. End-to-End Workflow: From Dataset Collection to Evaluation of LLama Models [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Dataset Sample B. Fine-Tuning We use LoRA (Low-Rank Adaptation) [44], a parameter￾efficient fine-tuning method enabling large pre-trained models to adapt with minimal computational cost. LoRA lowers memory usage while maintaining performance, allowing fine￾tuning on resource-limited hardware for large-scale tasks. Both models were trained for two epochs on a single A100 GPU, using a batch size of 32 and a maximum se… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 32 canonical work pages

  1. [1]

    A survey of large language models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023

    arXiv 2023

  2. [2]

    To what extent have llms reshaped the legal domain so far? a scoping literature review,

    B. Padiu, R. Iacob, T. Rebedea, and M. Dascalu, “To what extent have llms reshaped the legal domain so far? a scoping literature review,” Information, vol. 15, no. 11, p. 662, 2024

    work page 2024

  3. [3]

    Performance of advanced large language models (gpt-4o, gpt-4, gemini 1.5 pro, claude 3 opus) on japanese medical licensing examination: A comparative study,

    M. Liu, T. Okuhara, Z. Dai, W. Huang, H. Okada, F. Emi, and T. Kiuchi, “Performance of advanced large language models (gpt-4o, gpt-4, gemini 1.5 pro, claude 3 opus) on japanese medical licensing examination: A comparative study,” medRxiv, pp. 2024–07, 2024

    work page 2024

  4. [4]

    Large lan- guage models in patient education: a scoping review of applications in medicine,

    S. Aydin, M. Karabacak, V . Vlachos, and K. Margetis, “Large lan- guage models in patient education: a scoping review of applications in medicine,” Frontiers in Medicine, vol. 11, p. 1477898, 2024

    work page 2024

  5. [5]

    Google gemini as a next generation ai educational tool: a review of emerging educational technology,

    M. Imran and N. Almusharraf, “Google gemini as a next generation ai educational tool: a review of emerging educational technology,” Smart Learning Environments, vol. 11, no. 1, p. 22, 2024

    work page 2024

  6. [6]

    Large language models: A survey,

    S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Am- atriain, and J. Gao, “Large language models: A survey,” arXiv preprint arXiv:2402.06196, 2024

    arXiv 2024

  7. [7]

    A systematic comparison between open-and closed-source large language models in the context of gen- erating gdpr-compliant data categories for processing activity records,

    M. von Schwerin and M. Reichert, “A systematic comparison between open-and closed-source large language models in the context of gen- erating gdpr-compliant data categories for processing activity records,” Future Internet, vol. 16, no. 12, p. 459, 2024

    work page 2024

  8. [8]

    The llama 3 herd of models,

    A. G. et al., “The llama 3 herd of models,” 2024

    work page 2024

Show all 55 references
  1. [9]

    Phi-4 technical report,

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, J. R. Lee, Y . T. Lee, Y . Li, W. Liu, C. C. T. Mendes, A. Nguyen, E. Price, G. de Rosa, O. Saarikivi, A. Salim, S. Shah, X. Wang, R. Ward, Y . Wu, D. Yu, C....

    2024 arXiv

  2. [10]

    Larabench: Benchmarking arabic ai with large language models,

    A. Abdelali, H. Mubarak et al. , “Larabench: Benchmarking arabic ai with large language models,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Y . Graham and M. Purver, Eds. St. Julian’s,...

    2024

  3. [11]

    Parameter-efficient fine-tuning of large- scale pre-trained language models,

    N. Ding, Y . Qin, G. Yang, F. Wei, Z. Yang, Y . Su, S. Hu, Y . Chen, C.-M. Chan, W. Chen et al. , “Parameter-efficient fine-tuning of large- scale pre-trained language models,” Nature Machine Intelligence, vol. 5, no. 3, pp. 220–235, 2023

    2023

  4. [12]

    A comprehensive evaluation of large language models on legal judgment prediction,

    R. Shui, Y . Cao, X. Wang, and T.-S. Chua, “A comprehensive evaluation of large language models on legal judgment prediction,”

  5. [13]

    The role of legal expertise in interpretation of legal requirements and definitions,

    D. G. Gordon and T. D. Breaux, “The role of legal expertise in interpretation of legal requirements and definitions,” in 2014 IEEE 22nd International Requirements Engineering Conference (RE). IEEE, 2014, pp. 273–282

    2014

  6. [14]

    The impact of legal expertise on moral decision-making biases,

    S. Baez, M. Pati ˜no-S´aenz, J. Mart ´ınez-Cotrina, D. M. Aponte, J. C. Caicedo, H. Santamar ´ıa-Garc´ıa, D. Pastor, M. L. Gonz ´alez-Gadea, M. Haissiner, A. M. Garc ´ıa et al. , “The impact of legal expertise on moral decision-making biases,” Humanities and Social Sciences Co...

    2020

  7. [15]

    Deconfounding legal judgment prediction for European court of human rights cases towards better alignment with experts,

    S. T.y.s.s, S. Xu, O. Ichim, and M. Grabmair, “Deconfounding legal judgment prediction for European court of human rights cases towards better alignment with experts,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Y . Goldberg, Z. ...

    2022

  8. [16]

    A panoramic survey of natural language processing in the arab world,

    K. Darwish, N. Habash, M. Abbas, H. Al-Khalifa, H. T. Al-Natsheh, H. Bouamor, K. Bouzoubaa, V . Cavalli-Sforza, S. R. El-Beltagy, W. El- Hajj et al., “A panoramic survey of natural language processing in the arab world,” Communications of the ACM , vol. 64, no. 4, pp. 72–81, 2021

    2021

  9. [17]

    Native vs non-native language prompting: A comparative analysis,

    M. B. Kmainasi, R. Khan, A. E. Shahroor, B. Bendou, M. Hasanain, and F. Alam, “Native vs non-native language prompting: A comparative analysis,” arXiv preprint arXiv:2409.07054 , 2024

    2024 arXiv

  10. [18]

    A survey on legal judgment prediction: Datasets, metrics, models and challenges,

    J. Cui, X. Shen, F. Nie, Z. Wang, J. Wang, and Y . Chen, “A survey on legal judgment prediction: Datasets, metrics, models and challenges,”

  11. [19]

    Predicting supreme court decisions mathematically: A quantita- tive analysis of the “right to counsel

    F. Kort, “Predicting supreme court decisions mathematically: A quantita- tive analysis of the “right to counsel” cases,” American Political Science Review, vol. 51, no. 1, pp. 1–12, 1957

    1957

  12. [20]

    Legal judgment prediction based on machine learning: Predicting the discretionary damages of mental suffering in fatal car accident cases,

    D. Hsieh, L. Chen, and T. Sun, “Legal judgment prediction based on machine learning: Predicting the discretionary damages of mental suffering in fatal car accident cases,” Applied Sciences, vol. 11, no. 21, p. 10361, 2021

    2021

  13. [21]

    Neural legal judgment prediction in English,

    I. Chalkidis, I. Androutsopoulos, and N. Aletras, “Neural legal judgment prediction in English,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , A. Korhonen, D. Traum, and L. M `arquez, Eds. Florence, Italy: Association for Computat...

    2019

  14. [22]

    Recurrent neural networks (rnns): A gentle introduction and overview,

    R. M. Schmidt, “Recurrent neural networks (rnns): A gentle introduction and overview,” 2019. [Online]. Available: https://arxiv.org/abs/1912. 05911

    2019

  15. [23]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023. [Online]. Available: https://arxiv.org/abs/1706.03762

    2023 arXiv

  16. [24]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of naacL-HLT, vol. 1. Minneapolis, Minnesota, 2019, p. 2

    2019

  17. [26]

    Legal transformer models may not always help,

    S. Geng, R. Lebret, and K. Aberer, “Legal transformer models may not always help,” arXiv preprint arXiv:2109.06862 , 2021

    2021 arXiv

  18. [27]

    Pre-trained language models for the legal domain: a case study on indian law,

    S. Paul, A. Mandal, P. Goyal, and S. Ghosh, “Pre-trained language models for the legal domain: a case study on indian law,” in Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, 2023, pp. 187–196

    2023

  19. [28]

    Classifying european court of human rights cases using transformer-based techniques,

    A. S. Imran, H. Hodnefjeld, Z. Kastrati, N. Fatima, S. M. Daudpota, and M. A. Wani, “Classifying european court of human rights cases using transformer-based techniques,” IEEE Access, vol. 11, pp. 55 664–55 676, 2023

    2023

  20. [29]

    Transformer-based archi- tecture for judgment prediction and explanation in legal proceedings,

    A. Maqsood, A. Ul-Hasan, and F. Shafait, “Transformer-based archi- tecture for judgment prediction and explanation in legal proceedings,” in International Workshop on Document Analysis Systems . Springer, 2024, pp. 20–36

    2024

  21. [30]

    Fact-based court judgment prediction,

    S. K. Nigam and A. Deroy, “Fact-based court judgment prediction,” in Proceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation, 2023, pp. 78–82

    2023

  22. [31]

    ILDC for CJPE: Indian legal documents corpus for court judgment prediction and explanation,

    V . Malik, R. Sanjay, S. K. Nigam, K. Ghosh, S. K. Guha, A. Bhattacharya, and A. Modi, “ILDC for CJPE: Indian legal documents corpus for court judgment prediction and explanation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and t...

    2021

  23. [32]

    Evaluating transformer models for legal judgement prediction: A comparative study,

    T. Ghosh and S. Kumar, “Evaluating transformer models for legal judgement prediction: A comparative study,” in 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT). IEEE, 2024, pp. 1–4

    2024

  24. [33]

    Criminal court judgment prediction system built on modified bert models,

    S. Latisha, S. Favian, and D. Suhartono, “Criminal court judgment prediction system built on modified bert models,” Journal of Advances in Information Technology, vol. 15, no. 2, 2024

    2024

  25. [34]

    Not all languages are created equal in llms: Improving mul- tilingual capability by cross-lingual-thought prompting,

    H. Huang, T. Tang, D. Zhang, W. X. Zhao, T. Song, Y . Xia, and F. Wei, “Not all languages are created equal in llms: Improving mul- tilingual capability by cross-lingual-thought prompting,” arXiv preprint arXiv:2305.07004, 2023

    2023 arXiv

  26. [35]

    Democratizing llms for low-resource languages by leveraging their english dominant abilities with linguistically-diverse prompts,

    X.-P. Nguyen, S. M. Aljunied, S. Joty, and L. Bing, “Democratizing llms for low-resource languages by leveraging their english dominant abilities with linguistically-diverse prompts,” arXiv preprint arXiv:2306.11372 , 2023

    2023 arXiv

  27. [36]

    Llamalens: Specialized multilingual llm for analyzing news and social media content,

    M. B. Kmainasi, A. E. Shahroor, M. Hasanain, S. R. Laskar, N. Hassan, and F. Alam, “Llamalens: Specialized multilingual llm for analyzing news and social media content,” arXiv preprint arXiv:2410.15308, 2024

    2024 arXiv

  28. [37]

    Prediction of arabic legal rulings using large language models,

    A. Ammar, A. Koubaa, B. Benjdira, O. Nacar, and S. Sibaee, “Prediction of arabic legal rulings using large language models,”Electronics, vol. 13, no. 4, p. 764, 2024

    2024

  29. [38]

    Llms–the good, the bad or the indispensable?: A use case on legal statute prediction and legal judgment prediction on indian court cases,

    S. Vats, A. Zope, S. De, A. Sharma, U. Bhattacharya, S. K. Nigam, S. Guha, K. Rudra, and K. Ghosh, “Llms–the good, the bad or the indispensable?: A use case on legal statute prediction and legal judgment prediction on indian court cases,” in Findings of the Association for Com...

    2023

  30. [39]

    Legal judgment reimagined: PredEx and the rise of intelligent AI interpretation in Indian courts,

    S. K. Nigam, A. Sharma, D. Khanna, N. Shallum, K. Ghosh, and A. Bhattacharya, “Legal judgment reimagined: PredEx and the rise of intelligent AI interpretation in Indian courts,” in Findings of the Association for Computational Linguistics: ACL 2024 , L.-W. Ku, A. Martins, and ...

    2024

  31. [40]

    Gpt takes the bar exam,

    M. B. II and D. M. Katz, “Gpt takes the bar exam,” 2022. [Online]. Available: https://arxiv.org/abs/2212.14402

    2022 arXiv

  32. [41]

    Legal-bert: The muppets straight out of law school,

    I. Chalkidis, M. Fergadiotis, P. Malakasiotis, N. Aletras, and I. Androutsopoulos, “Legal-bert: The muppets straight out of law school,” 2020. [Online]. Available: https://arxiv.org/abs/2010.02559

    2020 arXiv

  33. [42]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” 2023. [Online]. Available: https://arxiv.org/abs/1910.10683

    2023 arXiv

  34. [43]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” 2019. [Online]. Available: https: //arxiv.org/abs/1908.10084

    2019 arXiv

  35. [44]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,”

  36. [45]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”

  37. [46]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81

    2004

  38. [47]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

    2002

  39. [48]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” 2020. [Online]. Available: https://arxiv.org/abs/1904.09675

    2020 arXiv

  40. [49]

    Are large language model-based evaluators the solution to scaling up multilingual evaluation?

    R. Hada, V . Gumma, A. Wynter, H. Diddee, M. Ahmed, M. Choudhury, K. Bali, and S. Sitaram, “Are large language model-based evaluators the solution to scaling up multilingual evaluation?” in Findings of the Association for Computational Linguistics: EACL 2024 , Y . Graham and M...

    2024

  41. [50]

    Can large language models be an alterna- tive to human evaluations?

    C.-H. Chiang and H.-y. Lee, “Can large language models be an alterna- tive to human evaluations?” arXiv preprint arXiv:2305.01937 , 2023

    2023 arXiv

  42. [51]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” 2023. [Online]. Available: https://arxiv.org/abs/2309.06180

    2023 arXiv

  43. [520]

    Available: https://aclanthology.org/2024.eacl-long.30

    [Online]. Available: https://aclanthology.org/2024.eacl-long.30

    2024

  44. [2017]

    Available: https://arxiv.org/abs/1412.6980

    [Online]. Available: https://arxiv.org/abs/1412.6980

    arXiv

  45. [2021]

    Available: https://arxiv.org/abs/2106.09685

    [Online]. Available: https://arxiv.org/abs/2106.09685

    arXiv

  46. [2022]

    Available: https://arxiv.org/abs/2204.04859

    [Online]. Available: https://arxiv.org/abs/2204.04859

    arXiv

  47. [2023]

    Available: https://arxiv.org/abs/2310.11761

    [Online]. Available: https://arxiv.org/abs/2310.11761

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.