Pith. sign in

REVIEW 4 major objections 5 minor 57 references

Bilingual BSARD: Extending Statutory Article Retrieval to Dutch

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper presents bBSARD, a parallel French–Dutch statutory article retrieval benchmark for Belgian law, and shows that fine-tuning small language-specific models matches or beats proprietary zero-shot embedding models.

desk verdict A useful bilingual legal retrieval benchmark with a real translation-validation gap; worth engaging, but the Dutch-side numbers need more verification. read the letter →

arxiv 2412.07462 v1 pith:CJVVVMY2 submitted 2024-12-10 cs.CL cs.IR

Ehsan Lotfi , Nikolay Banar , Nerses Yuzbashyan , Walter Daelemans This is my paper · ORCID
classification cs.CLcs.IR
keywords legalinformationretrievalstatutoryarticlemultilingualDutchlanguageFrenchparallelcorpusdenseBM25
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces bBSARD, a bilingual extension of the French BSARD legal retrieval dataset, adding Dutch translations of Belgian statutory articles and of the dataset's 1,108 legal questions. It argues that bBSARD fills a gap by providing a public retrieval benchmark for Dutch statutory law, enabling reliable evaluation of Dutch retrieval models in the legal domain. On this benchmark, the paper reports that BM25 remains a competitive baseline against many zero-shot dense models, that proprietary embedding APIs lead the zero-shot setting, and that fine-tuning small language-specific models such as RobBERT-2023 and FlauBERT can match or surpass those proprietary results. The paper also documents a consistent performance advantage for French over Dutch, which it attributes tentatively to differences in pre-training data availability. If the benchmark is sound, it gives the community a reproducible testbed for improving Dutch legal search and for studying cross-lingual transfer between French and Dutch.

What carries the argument

The central object is the bBSARD dataset itself: a parallel corpus of 22,417 Belgian statutory articles aligned in French and Dutch by law code and article number, paired with 1,108 legal questions (886 train, 222 test) whose Dutch translations were produced by GPT-4o with the original articles as context, and whose relevance labels are inherited from BSARD's expert annotations. The mechanism that produces the paper's headline result is the Siamese bi-encoder fine-tuning protocol—queries and articles encoded by the same transformer, trained with contrastive loss, temperature 0.05, and in-batch negatives—which lets small language-specific models such as RobBERT-2023 and FlauBERT surpass much larger zero-shot models on ranking metrics.

What would settle it

Have two qualified legal translators independently translate all 1,108 questions from French to Dutch and compare against GPT-4o's translations, then re-run the benchmark on the corrected translations; if the corrected translations change which articles are relevant, the Dutch-side results and model rankings in the paper are not reliable.

Watch

Extended reading notes

Core claim

The central claim is that bBSARD is a valid benchmark for statutory article retrieval in Dutch and French, constructed by scraping and aligning 22,417 parallel Belgian law articles (99% of the BSARD corpus) from the Justel database and translating BSARD's 1,108 expert-labeled legal questions from French to Dutch with GPT-4o at temperature 0, followed by human inspection of a 100-question sample. The paper's benchmarking over lexical models (TF-IDF, BM25), context-independent embeddings, zero-shot dense models (open and proprietary), and fine-tuned small foundation models leads to three findings: (1) BM25 is competitive with zero-shot dense models below 300M parameters; (2) proprietary models (voyage-3, text-embedding-3-large) are the best zero-shot performers; and (3) fine-tuned small language-specific models—RobBERT-2023 for Dutch and FlauBERT for French—match or exceed those proprietary models on ranking metrics. The paper also finds that French retrieval consistently outperforms Dutch across most model types, and observes preliminary evidence that training on combined French and Dutch data improves a multilingual model's Dutch performance.

Load-bearing premise

The Dutch evaluation rests on the assumption that GPT-4o's translations of the 1,108 legal questions preserve the legal meaning closely enough that BSARD's expert relevance labels still apply to the Dutch corpus; only 100 translations were human-checked, with 2% legally inaccurate and 6% minor issues, leaving 91% unverified.

Editorial extensions

If this is right

  • Dutch legal information retrieval now has a public, reproducible benchmark with expert relevance labels, so future Dutch retrieval models can be evaluated on a common test set.
  • Fine-tuning small language-specific models offers a cost-effective alternative to proprietary embedding APIs for legal retrieval, achieving higher MAP, MRR, and nDCG on both Dutch and French.
  • BM25 should be kept as a standard baseline in future legal-retrieval evaluations, since it outperforms many dense models in the zero-shot setting.
  • The parallel French–Dutch structure enables cross-lingual transfer experiments; the paper's observation that XLM-RoBERTa improves in Dutch when trained on mixed-language data suggests bilingual training can benefit the lower-resource language.
  • The consistent French-over-Dutch performance gap points to pre-training data volume as a likely factor, motivating investment in Dutch corpora and Dutch-specific models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the unverified 91% of Dutch translations contain systematic legal errors at a rate similar to the 100-question sample (2% legally inaccurate, 6% minor), then the Dutch subset's relevance labels are noisy, and any conclusions about the exact ordering of models on Dutch should be read with caution until the translations are fully corrected.
  • The parallel nature of bBSARD could also support adversarial robustness studies—for example, testing whether a model trained only on French articles can retrieve Dutch articles via cross-lingual transfer, which the paper only begins to explore.
  • A natural next experiment would be to add generative or LLM-based re-rankers to the bBSARD leaderboard; the paper's benchmark currently only covers bi-encoders and lexical models, leaving out a class of methods now common in retrieval pipelines.
  • The French-over-Dutch gap, if driven by pre-training data size, implies that investing in Dutch pre-training corpora is likely to narrow the gap; this is a testable prediction that could be checked by comparing models pre-trained on different Dutch data volumes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces bBSARD, a Dutch extension of the French BSARD statutory article retrieval dataset. The authors scrape parallel French and Dutch Belgian legislation from the Justel portal (22,417 aligned articles), translate the 1,108 BSARD questions into Dutch with GPT-4o, and human-check 100 of the translations. They then benchmark lexical models (TF-IDF, BM25), zero-shot dense models (open and proprietary), and fine-tuned small language models (RobBERT, Tik-to-Tok, CamemBERT, FlauBERT, XLM-R) on both language subsets. The reported findings are that BM25 remains a strong baseline, proprietary embedding models lead the zero-shot setting, and fine-tuned language-specific small models can match or surpass those proprietary models; French results are on average higher than Dutch results. The dataset and evaluation code are publicly released.

Significance. If the resource is valid, bBSARD fills a genuine gap: there is no public Dutch statutory article retrieval benchmark with expert relevance labels, and the parallel French--Dutch corpus enables cross-lingual retrieval studies. The evaluation is broad, covering 19 models across lexical, zero-shot dense, and fine-tuned settings, and the public release of data and code is a concrete community asset. The paper is also transparent about corpus coverage and about the fact that the Dutch question set is machine-translated. However, the validity of the Dutch benchmark hinges on the assumption that the translated questions preserve the French relevance labels; the evidence for that assumption is currently thin, and a few reproducibility details are missing. These issues are fixable and do not appear to require a change in the overall approach, but they need to be addressed before the resource can be used with confidence.

major comments (4)
  1. [Section 3.2] The Dutch side of bBSARD inherits BSARD relevance labels solely through translation of the 1,108 questions, but only 100 translations (9%) were human-checked. The check already found 2% legally inaccurate and 6% minor semantic/grammatical/lexical issues, and the error rate and error types in the remaining 91% are unknown. If the unverified translations contain systematic legal-terminology errors, both fine-tuning (noisy training queries) and evaluation (wrong gold articles) on the Dutch subset would be corrupted. The paper itself notes in Section 5.3 that translation issues may particularly affect lexical models, and the Conclusions state that future work will 'improve bBSARD's quality by manually checking/correcting all translated questions,' which confirms that the current release is not fully verified. Please either verify and correct all question translations before release, or provide additional systematic evidence (e.g., full back-translation with legal review, error analysis by legal subdomain, inter-annotator agreement) that the inherited labels remain valid.
  2. [Section 4.1.2] The paper does not specify how chunk-level scores are aggregated into article-level scores for models with a 512-token input limit. Section 4.1.2 says that text is divided into overlapping chunks of 200 tokens with 20-token overlap, but it never states whether the article score is the maximum, mean, or some other aggregate of the chunk scores. Since all dense-model results in Tables 1 and 2 depend on this choice, the benchmark is not reproducible without this information. Please state the exact aggregation rule and, ideally, release the scoring script.
  3. [Section 5.1 / Table 1] The claim that 'zero-shot dense models only fully outperforming [BM25] starting from 300 million parameters' is contradicted by Table 1: mE5base (278M parameters) exceeds BM25 on all seven reported metrics (e.g., R@100 50.14 vs. 40.19, MAP@100 16.47 vs. 16.07, MRR@100 25.64 vs. 22.63). The threshold should be corrected or the claim qualified (for example, 'clearly' or 'consistently'), and the analogous statement for the French subset should be rechecked against Table 2.
  4. [Section 6] The Conclusions report an experimental observation that XLM-Roberta performs better in Dutch when fine-tuned for 50+50 epochs on French and Dutch data than when fine-tuned for 100 epochs on Dutch data. No details of this experiment appear in Section 4 or in the results tables: the data-mixing procedure, hyperparameters, and metric values are all unspecified. Since this is a non-trivial cross-lingual training claim, it should either be presented with full experimental details and results in a table or removed.
minor comments (5)
  1. [Section 5.2 / Table 2] Comparative statements such as 'FlauBERTbase beating voyage-3 in 4 out of 7 metrics' are based on a single evaluation run on a 222-question test set. Please add confidence intervals, bootstrap estimates, or multiple-seed runs so that small differences are not over-interpreted.
  2. [Section 3.2] The translation check was performed by a single 'native speaker'; it is not stated whether this annotator has legal expertise. For legal terminology, legal-expert annotation or at least a second annotator would strengthen the quality claim.
  3. [Appendix A] The alignment pipeline is described as using 'ChatGPT queries' without specifying the model version or prompt. Please provide the exact model and protocol so that the alignment process is reproducible.
  4. [Table 3] The bBSARD French article counts differ slightly from the original BSARD counts for several codes (e.g., Electoral Code 218 vs. 217, Code of Various Rights and Taxes 191 vs. 189). A sentence explaining whether these differences affect comparability with the original BSARD results would be helpful.
  5. [Limitations] There is a typo: 'obtaining any comprehensive legal information or advise' should read 'advice.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circular dependency found: bBSARD is an external-data extension benchmark with train/test splitting and independent evaluation, and its stated limitations concern translation quality, not derivation circularity.

full rationale

The paper's central claim is the construction of a bilingual retrieval dataset by scraping parallel Dutch and French articles from the Justel database and translating the BSARD questions into Dutch with GPT-4o; no quantity is derived from another by construction. Article alignment is verified against the official BSARD corpus (22,417 of 22,633 articles recovered), and the question-to-article relevance labels are inherited from BSARD's expert annotations rather than predicted by any model in this paper. All retrieval evaluations use the fixed BSARD train/test split (886/222), with fine-tuned models trained only on the training split and evaluated on held-out test questions, so the reported comparisons are not fitted-input predictions. The citations to BSARD (Louis and Spanakis, 2022) reference an externally published dataset that is the input resource, not an unverified premise of this paper, and the claimed contribution is the new Dutch-side resource and benchmark, which is independently testable. The acknowledged limitation that only 100 of 1,108 translated questions were human-checked (2% legally inaccurate and 6% minor issues) is a validity caveat about translation quality, explicitly acknowledged in Section 3.2 and revisited in Section 5.3; it does not make any model result equivalent to any model input and is not a circularity. No equation, parameter, or prediction in the paper reduces to its own inputs, and no self-citation chain is load-bearing for the benchmark's conclusions. Therefore the correct finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central contribution is empirical, not a derivation: the paper inherits BSARD's annotations and external model outputs. Its trustworthiness rests on domain assumptions about translation fidelity, legislative alignment, chunking, and the adequacy of single-run metric estimates.

free parameters (3)
  • chunk_size = 200 tokens, 20-token overlap
    Chosen for models with 512-token input limits; affects how long articles are represented and therefore retrieval scores; no sensitivity analysis is reported.
  • translation_temperature = 0
    GPT-4o translation setting; deterministic but not independently validated for all questions.
  • fine-tuning_hyperparameters = lr=2e-5, batch_size=22, temperature=0.05, 100 epochs
    Follows the BSARD setup; these choices affect fine-tuned model rankings but are not fitted to the test set.
assumptions (4)
  • domain assumption BSARD relevance labels remain valid for the Dutch translations and aligned Dutch articles.
    The benchmark treats article IDs assigned to French questions as correct for the Dutch versions; any translation shift in legal meaning breaks this, as noted in Section 3.2 and Limitations.
  • domain assumption The Justel web pages provide correctly aligned French and Dutch versions of the same legislative text at the same enforcement date.
    The alignment procedure relies on article codes, a length filter, ChatGPT queries, and manual fixes; small discrepancies (e.g., Art. 14.2 vs 14/2) are acknowledged in Section 3.1 and Appendix A.
  • domain assumption Chunking documents into 200-token windows with 20-token overlap preserves the information needed for relevant-article ranking.
    Used for all zero-shot models with 512-token input limits; no analysis of aggregation or alternative chunk sizes is reported in Section 4.1.2.
  • domain assumption Standard IR metrics on a single test split are sufficient to rank model families.
    Claims about which model wins rely on point estimates with no confidence intervals, as visible throughout Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bilingual BSARD: Extending Statutory Article Retrieval to Dutch." pith.science (2026). https://pith.science/paper/CJVVVMY2

@misc{pith2026241207462,
  author       = {Pith},
  title        = {Pith review of: Bilingual BSARD: Extending Statutory Article Retrieval to Dutch},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CJVVVMY2}},
  note         = {Machine review of arXiv:2412.07462}
}
read the original abstract

Statutory article retrieval plays a crucial role in making legal information more accessible to both laypeople and legal professionals. Multilingual countries like Belgium present unique challenges for retrieval models due to the need for handling legal issues in multiple languages. Building on the Belgian Statutory Article Retrieval Dataset (BSARD) in French, we introduce the bilingual version of this dataset, bBSARD. The dataset contains parallel Belgian statutory articles in both French and Dutch, along with legal questions from BSARD and their Dutch translation. Using bBSARD, we conduct extensive benchmarking of retrieval models available for Dutch and French. Our benchmarking setup includes lexical models, zero-shot dense models, and fine-tuned small foundation models. Our experiments show that BM25 remains a competitive baseline compared to many zero-shot dense models in both languages. We also observe that while proprietary models outperform open alternatives in the zero-shot setting, they can be matched or surpassed by fine-tuning small language-specific models. Our dataset and evaluation code are publicly available.

Figures

Figures reproduced from arXiv: 2412.07462 by the authors.

Figure 1
Figure 1. Distribution of different codes in the bBSARD article corpus. ’Relevant’ articles (green) are the ones cited [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Basic statistics of bBSARD. From the left: Number of words in the articles (French and Dutch), Number [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Standard Siamese Bi-Encoder setting with in-batch negatives, which we use for fine-tuning. Articles and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 20 canonical work pages

  1. [1]

    Charu C Aggarwal and ChengXiang Zhai. 2012. A survey of text clustering algorithms. Mining text data, pages 77--128

    work page 2012

  2. [2]

    Paheli Bhattacharya, Kripabandhu Ghosh, Saptarshi Ghosh, Arindam Pal, Parth Mehta, Arnab Bhattacharya, and Prasenjit Majumder. 2019. Fire 2019 aila track: Artificial intelligence for legal assistance. In Proceedings of the 11th annual meeting of the forum for information retrieval evaluation, pages 4--6

    work page 2019

  3. [3]

    Paheli Bhattacharya, Parth Mehta, Kripabandhu Ghosh, Saptarshi Ghosh, Arindam Pal, Arnab Bhattacharya, and Prasenjit Majumder. 2021. https://doi.org/10.1145/3441501.3441510 Fire 2020 aila track: Artificial intelligence for legal assistance . In Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE '20, page 1–3, Ne...

    arXiv 2021

  4. [4]

    Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the association for computational linguistics, 5:135--146

    2017

  5. [5]

    Marius B \"u ttner and Ivan Habernal. 2024. Answering legal questions from laymen in german civil law system. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2015--2027

    work page 2024

  6. [6]

    Andong Chen, Feng Yao, Xinyan Zhao, Yating Zhang, Changlong Sun, Yun Liu, and Weixing Shen. 2023. Equals: A real-world dataset for legal question answering via reading chinese laws. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pages 71--80

    2023

  7. [7]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://arxiv.org/abs/2402.03216 Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . Preprint, arXiv:2402.03216

    arXiv 2024

  8. [8]

    Pieter Delobelle and Fran c ois Remy. 2024. Robbert-2023: Keeping dutch language models up-to-date at a lower cost thanks to model conversion. Computational Linguistics in the Netherlands Journal, 13:193--203

    work page 2024

Show all 57 references
  1. [9]

    Pieter Delobelle, Thomas Winters, and Bettina Berendt. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.292 R ob BERT : a D utch R o BERT a-based L anguage M odel . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3255--3265, Online. Associa...

    2020 doi

  2. [10]

    European Union . 2012. Charter of fundamental rights of the european union. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:12012P/TXT. [Accessed: 5 October 2024]

    2012

  3. [11]

    Jean-Philippe Fauconnier. 2015. http://fauconnier.github.io French word embeddings

    2015

  4. [12]

    Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic bert sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878--891

    2022

  5. [13]

    Gregor Geigle, Nils Reimers, Andreas R \"u ckl \'e , and Iryna Gurevych. 2021. Tweac: transformer with extendable qa agent classifiers. arXiv preprint arXiv:2104.07081

    2021 arXiv

  6. [14]

    Randy Goebel, Yoshinobu Kano, Mi-Young Kim, Juliano Rabelo, Ken Satoh, and Masaharu Yoshioka. 2024. Overview and discussion of the competition on legal information, extraction/entailment (coliee) 2023. The Review of Socionetwork Strategies, 18(1):27--47

    2024

  7. [15]

    \'E douard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tom \'a s Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

    2018

  8. [16]

    Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-Hsuan Sung, L \'a szl \'o Luk \'a cs, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart reply. arXiv preprint arXiv:1705.00652

    2017 arXiv

  9. [17]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised dense information retrieval with contrastive learning. Transactions on Machine Learning Research

    2022

  10. [18]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)...

    2020

  11. [19]

    Mi-Young Kim, Juliano Rabelo, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. 2022. Coliee 2022 summary: methods for legal document retrieval and entailment. In JSAI International Symposium on Artificial Intelligence, pages 51--67. Springer

    2022

  12. [20]

    Hang Le, Lo\" i c Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Beno\^ i t Crabb\' e , Laurent Besacier, and Didier Schwab. 2020. https://www.aclweb.org/anthology/2020.lrec-1.302 Flaubert: Unsupervised language model pre-training ...

    2020

  13. [21]

    Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024. Making text embedders few-shot learners. arXiv preprint arXiv:2409.15700

    2024 arXiv

  14. [22]

    Ilya Loshchilov and Frank Hutter. 2017. https://api.semanticscholar.org/CorpusID:53592270 Decoupled weight decay regularization . In International Conference on Learning Representations

    2017

  15. [23]

    Antoine Louis, Vageesh Saxena, Gijs van Dijck, and Gerasimos Spanakis. 2024 a . Colbert-xm: A modular multi-vector representation model for zero-shot multilingual information retrieval. arXiv preprint arXiv:2402.15059

    2024 arXiv

  16. [24]

    Antoine Louis and Gerasimos Spanakis. 2022. A statutory article retrieval dataset in french. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6789--6803

    2022

  17. [25]

    Antoine Louis, Gijs van Dijck, and Gerasimos Spanakis. 2024 b . Interpretable long-form legal question answering with retrieval-augmented large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 22266--22275

    2024

  18. [26]

    Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142--150

    2011

  19. [27]

    Louis Martin, Benjamin Muller, Pedro Javier Ortiz Su \'a rez, Yoann Dupont, Laurent Romary, \'E ric Villemonte de la Clergerie, Djam \'e Seddah, and Beno \^ t Sagot. 2020. Camembert: a tasty french language model. In Proceedings of the 58th Annual Meeting of the Association fo...

    2020

  20. [28]

    Corrado, and Jeffrey Dean

    Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013 a . https://api.semanticscholar.org/CorpusID:5959482 Efficient estimation of word representations in vector space . In International Conference on Learning Representations

    2013

  21. [29]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013 b . Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26

    2013

  22. [30]

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. Mteb: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014--2037

    2023

  23. [31]

    OpenAI. 2024. https://openai.com/index/new-embedding-models-and-api-updates/ New embedding models and api updates . Accessed: 2024-10-31

    2024

  24. [32]

    James O’Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, and Danushka Bollegala. 2021. I wish i would have loved this one, but i didn’t--a multilingual dataset for counterfactual detection in product review. In Proceedings of the 2021 Conference on Empirical Methods in...

    2021

  25. [33]

    Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.373 Towards making the most of C hat GPT for machine translation . In Findings of the Association for Computational Ling...

    2023 doi

  26. [34]

    Juliano Rabelo, Randy Goebel, Mi-Young Kim, Yoshinobu Kano, Masaharu Yoshioka, and Ken Satoh. 2022. Overview and discussion of the competition on legal information extraction/entailment (coliee) 2021. The Review of Socionetwork Strategies, 16(1):111--133

    2022

  27. [35]

    Juliano Rabelo, Mi-Young Kim, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. 2021. Coliee 2020: methods for legal document retrieval and entailment. In New Frontiers in Artificial Intelligence: JSAI-isAI 2020 Workshops, JURISIN, LENLS 2020 Workshops, Virtual E...

    2021

  28. [36]

    Nils Reimers and Iryna Gurevych. 2019. https://api.semanticscholar.org/CorpusID:201646309 Sentence-bert: Sentence embeddings using siamese bert-networks . In Conference on Empirical Methods in Natural Language Processing

    2019

  29. [37]

    Fran c ois Remy, Pieter Delobelle, Bettina Berendt, Kris Demuynck, and Thomas Demeester. 2023. Tik-to-tok: Translating language models one token at a time: An embedding initialization strategy for efficient language adaptation. arXiv preprint arXiv:2310.03477

    2023 arXiv

  30. [38]

    Robertson, Steve Walker, Susan Jones, Micheline Hancock - Beaulieu, and Mike Gatford

    Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock - Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3 . In Proceedings of The Third Text REtrieval Conference, TREC 1994, Gaithersburg, Maryland, USA, November 2-4, 1994 , volume 500-225 of NIST Special Publicati...

    1994

  31. [39]

    Gerard Salton and Chung-Shu Yang. 1973. On the specification of term values in automatic indexing. Journal of documentation, 29(4):351--372

    1973

  32. [40]

    Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018. Carer: Contextualized affect representations for emotion recognition. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 3687--3697

    2018

  33. [41]

    Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation, 28(1):11--21

    1972

  34. [42]

    Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael G \"u nther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, et al. 2024. jina-embeddings-v3: Multilingual embeddings with task lora. arXiv preprint arXiv:2409.10173

    2024 arXiv

  35. [43]

    Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Zibing Que, Ning Zheng, Yun Liu, Weixing Shen, and Yiqun Liu. 2024. Stard: A chinese statute retrieval dataset with real queries issued by non-professionals. arXiv preprint arXiv:2406.15313

    2024 arXiv

  36. [44]

    Nandan Thakur, Nils Reimers, Andreas R \"u ckl \'e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://openreview.net/forum?id=wCu6T5xFjeJ BEIR : A heterogeneous benchmark for zero-shot evaluation of information retrieval models . In Thirty-fifth Conference on Neural Info...

    2021

  37. [45]

    St \'e phan Tulkens, Chris Emmery, and Walter Daelemans. 2016. Evaluating unsupervised dutch word embeddings as a linguistic resource. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 4130--4136

    2016

  38. [46]

    VoyageAI. 2024. https://blog.voyageai.com/2024/09/18/voyage-3/ Voyage 3 . Accessed: 2024-10-31

    2024

  39. [47]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533

    2022 arXiv

  40. [48]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368

    2023 arXiv

  41. [49]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672

    2024 arXiv

  42. [50]

    Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al. 2018. Cail2018: A large-scale legal dataset for judgment prediction. arXiv preprint arXiv:1807.02478

    2018 arXiv

  43. [51]

    Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Tianyang Zhang, Xianpei Han, Zhen Hu, Heng Wang, et al. 2019. Cail2019-scm: A dataset of similar case matching in legal domain. arXiv e-prints, pages arXiv--1911

    2019

  44. [52]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. https://arxiv.org/abs/2407.19669 mgte: Generalized long-context text representation and reranking models for ...

    2024 arXiv

  45. [53]

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. 2024. Dense text retrieval based on pretrained language models: A survey. ACM Transactions on Information Systems, 42(4):1--60

    2024

  46. [54]

    Haoxi Zhong, Chaojun Xiao, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al. 2018. Overview of cail2018: Legal judgment prediction competition. arXiv preprint arXiv:1810.05851

    2018 arXiv

  47. [55]

    Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. Jec-qa: a legal-domain question answering dataset. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9701--9708

    2020

  48. [56]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.