Pith. sign in

REVIEW 5 major objections 5 minor 20 references

Analyzing Bias in Swiss Federal Supreme Court Judgments Using Facebook's Holistic Bias Dataset: Implications for Language Model Training

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that what looks like social bias in Swiss Federal Supreme Court judgments—descriptors like 'victime' and 'Opfer' co-occurring with dismissal—is actually neutral legal language, a language artifact rather than a social bias.

desk verdict The paper's honest conclusion (the 'bias' is a language artifact) is its real contribution, but the abstract and tables overstate what the analysis establishes. read the letter →

arxiv 2501.03324 v1 pith:I35URBTH submitted 2025-01-06 cs.CL cs.AI

Sabine Wehnert , Muhammet Ertas , Ernesto William De Luca This is my paper · ORCID
classification cs.CLcs.AI
keywords biasanalysislegaljudgmentpredictionSwissdatasetbinomialsignificancetestattentionvisualizationlanguageartifactsmultilingualtextHolisticdescriptors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a descriptor-based bias audit of the Swiss Judgment Prediction dataset, using the Holistic Bias dataset's 'dispreferred' descriptors translated into German, French, and Italian, surfaces statistically reliable associations with dismissal and approval labels. It further claims that these associations propagate into the behavior of a fine-tuned legal language model, which attends to words like 'victime' and 'Opfer' when predicting dismissal. But the paper's central, carefully qualified conclusion is negative: on inspection, these are language artifacts, not social bias, because words like 'victim' are ordinary, neutral legal terminology in Swiss rulings. The stakes are practical: if true, a naive cross-lingual bias audit of legal text produces false positives that could mislead efforts to debias training data.

What carries the argument

The two load-bearing tools are the binomial significance test and attention visualization. The binomial test compares the observed count of a descriptor co-occurring with 'dismissal' or 'approval' against the null hypothesis that the label follows the dataset's base rate ($\pi_0 = 0.762$ for dismissal), flagging any descriptor whose p-value falls below a lenient threshold of 0.1. Attention visualization then computes token-level attribution scores on the fine-tuned legal BERT model, showing which descriptor tokens most influence the prediction; a score near 1 indicates strong positive influence toward the predicted class. Together these tools let the paper show both that the association exists in the data and that the model has learned to rely on it.

What would settle it

Compare the co-occurrence of 'victime' and 'Opfer' with negative outcomes in the Swiss court facts against a matched non-legal Swiss corpus with a similar negative-outcome base rate; if the association disappears outside legal text, the claim that this is a legal-register artifact rather than social bias is supported. Alternatively, retrain the model with these tokens replaced by neutral legal synonyms and check whether the systematic dismissal bias in attention attributions disappears.

Watch

Extended reading notes

Core claim

The central claim is that the binomial significance test identifies a set of 'dispreferred' descriptors whose co-occurrence with the dismissal label deviates significantly from the dataset's 76.2 percent base rate—most notably 'victime' (p = 8.36e-08), 'intitulé', 'berechtigt', 'Opfer', and 'menacé' for dismissal, and 'en danger', 'Hausfrau', 'Behinderte', 'behindert', and 'délicate' for approval. Attention analysis on the fine-tuned model shows that among these, 'victime' and 'Opfer' consistently pull predictions toward 'dismissal', even when the true label is approval. The paper's own conclusion states that instead of finding social bias, it observed language artifacts for the translations of the descriptor 'victim', which is neutral in a legal context. Thus the discovery is a cautionary one: a statistically significant, model-influencing association can arise entirely from legal register rather than from social stereotyping.

Load-bearing premise

The assumption that a word that is socially loaded in English remains socially loaded after machine translation into German, French, and Italian legal text, which the paper itself concedes is doubtful in Section 4.3.

Editorial extensions

If this is right

  • If the paper's conclusion is right, descriptor-based bias audits cannot be transferred across languages or domains without first validating that the descriptor's social connotation survives translation into the target legal register.
  • The same pipeline applied to other multilingual legal corpora would likely produce similar false positives, meaning reported 'bias' in legal NLP datasets should be scrutinized for register artifacts before acting on it.
  • Removing 'victime', 'Opfer', or 'berechtigt' from training data to eliminate apparent bias could harm model performance, because these words carry genuinely informative legal meaning about the case.
  • The combination of binomial testing and attention attribution remains a usable diagnostic, but only with descriptors that are context-validated rather than directly ported from English.
  • The paper's distinction between social bias and language artifacts gives legal NLP researchers a concrete example of why statistical significance alone is not evidence of unfairness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A decisive follow-up would replace 'victime' and 'Opfer' with neutral legal synonyms in a held-out test set and measure whether the dismissal-prediction bias disappears; the paper does not run this causal intervention, only correlational attention analysis.
  • The paper's use of a 0.1 significance threshold is unusually lenient; under a conventional 0.01 or 0.05 threshold the list of 'biased' descriptors shrinks considerably, suggesting the evidence for even the language-artifact pattern is thinner than the headline counts imply.
  • The core lesson generalizes beyond law: any bias vocabulary defined in one cultural context and machine-translated into another needs an in-context validation step, otherwise audits will confound register, domain, and social stigma.
  • The finding hints that legal language models fine-tuned on Swiss rulings are not learning social stereotypes from these descriptors, but they are learning a legally legitimate cue; future fairness work should therefore focus on genuinely demographic language, such as party names or nationality mentions, rather than victim-related terms.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper analyzes the Swiss Judgment Prediction Dataset (SJP) for social bias by taking the 'dispreferred' descriptors from Facebook's Holistic Bias dataset, translating them into German, French, and Italian, and counting their co-occurrence with judicial outcomes ('dismissal' vs. 'approval'). The authors apply a binomial significance test to identify descriptors whose outcome rates deviate from the corpus base rate, fine-tune legal-swiss-roberta-large models on summarized and chunked versions of the data, and use attention visualizations to examine whether certain descriptors influence model predictions. The paper reports a set of 'biased' descriptors in Table 7 and presents attention evidence for 'victime' and 'Opfer'. However, the authors themselves conclude in Section 5 that the main finding is a language artifact rather than social bias, because victim-related words are neutral legal terms.

Significance. The paper's descriptive contribution is useful: it documents co-occurrence counts for translated Holistic Bias descriptors in a large multilingual legal corpus, and it makes code available for reproducibility. The attention analysis, while exploratory, is a reasonable first step toward linking corpus statistics to model behavior. The honest limitation discussion in Section 4.3 is a strength. However, the inferential claims are not established. The binomial test at alpha=0.1 without multiple-testing correction, the absence of a neutral legal-term control, and the reliance on two attention examples mean the central 'bias' claim is not supported as stated. The paper's own conclusion narrows the contribution to a language-artifact finding, which is more modest than the abstract and title suggest.

major comments (5)
  1. [Section 3.5.3, Tables 5-7] The significance threshold alpha=0.1 is applied to each descriptor-outcome pair independently. With 70 descriptors and two outcomes, roughly 14 null rejections are expected by chance alone. The reported p-values include several near 0.01, which would not survive even a simple Bonferroni or Benjamini-Hochberg correction. The paper should report adjusted p-values or q-values, or justify why correction is unnecessary. As written, the list in Table 7 is likely to contain false positives.
  2. [Section 3.2, Section 4.3, Table 5] The BST result for 'victime' (3,132 dismissals out of 3,928, p=8.36e-08) is a deviation from the base rate of 0.762, but the design has no control condition that would distinguish social bias from legal register. The paper's own limitation section and conclusion acknowledge that 'victime', 'Opfer', and 'berechtigt' are neutral legal terms. Since no frequency-matched control set of ordinary legal terms (e.g., 'recours', 'Beschwerde', 'requérant') is tested, the same BST would likely flag many non-descriptor legal terms as 'biased'. This missing control is load-bearing: without it, the paper cannot claim that Holistic Bias descriptors specifically reveal bias rather than simply reflecting case-type language.
  3. [Section 4.2, Figures 7-8] The attention analysis is anecdotal. The claim that 'victime' and 'Opfer' show a consistent pattern is based on two example visualizations, while the broader analysis of top-attribution counts (top 50 and top 20) does not compare against a baseline distribution or random token attributions. No quantitative metric, such as the proportion of occurrences with high attribution or a permutation test, is provided. Consequently, the conclusion that these descriptors 'may tend to make biased predictions' is not supported with the evidence shown.
  4. [Section 3.6, Section 4.2] The choice of seed 48 for the chunked model is described as the first model with balanced predictions, made after inspecting results. This is a post hoc model-selection procedure, and the reported attention results come from that selected model only. Because the selection criterion is tied to prediction balance rather than to a pre-registered analysis, the results are at risk of selection bias. The paper should either report results across all seeds or justify the selection criterion independently of the outcome.
  5. [Section 4.1, Table 7] The descriptors in Table 7 include morphological variants and translations of the same English descriptor (e.g., 'victime', 'Opfer', 'vittima'; 'menacé', 'menacée', 'bedroht', 'in pericolo'). These are counted as separate tokens but are not statistically independent, and a single court case can contain multiple descriptor forms. The BST treats each descriptor occurrence as an independent observation, which inflates significance. The analysis should cluster by case or at least report the number of distinct cases contributing to each descriptor count.
minor comments (5)
  1. [Abstract and Section 5] The abstract says the study 'identifies biases', while the conclusion says that instead of social bias, the authors observed language artifacts for translations of 'victim'. The framing should be aligned; the current wording overstates what the analysis establishes.
  2. [Table 2] The weighted-average F1-score for seed 48 after extractive summarization is reported as 0.93, but with precision 0.81 and recall 0.78 the correct value is approximately 0.79. This appears to be a typographical error.
  3. [Section 5] The conclusion contains a duplicated word: 'translations of of the descriptor “victim”'.
  4. [Tables 1-4] The 'Support' values are written with a decimal point (14.026) where the thousands separator is likely intended (14,026). This should be corrected to avoid confusion.
  5. [Section 3.6] The tie-breaking rule for chunked predictions (always selecting 'dismissal') is stated but not analyzed. Since the chunking model already favors the majority class, this rule may further distort the descriptor-prediction analysis. A sensitivity analysis with the opposite tie-break rule would be informative.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the analysis uses externally sourced descriptors, a standard binomial test against the observed base rate, and attention measurements on held-out test predictions, so no result reduces to its own inputs.

full rationale

The derivation chain runs from externally sourced Holistic Bias descriptors (Section 3.1), through translation and derivation (Section 3.2), to a binomial test comparing token-specific label frequencies to the corpus base rate (Section 3.5), and finally to fine-tuning an external legal BERT model and inspecting attention on predicted test facts (Sections 3.6 and 4.2). None of these steps defines its output in terms of its input: the binomial test is a standard goodness-of-fit comparison, not a fitted parameter later relabeled as a prediction, and the attention analysis uses held-out model predictions rather than the same training instances that produced the descriptor lists. The paper's self-citations [14, 18] are to prior applications of the same standard statistical test and to earlier sentiment work on the same benchmark; neither supplies an unverified theorem or a forced ansatz that the central conclusions depend on. The authors' own limitation section and conclusion concede that the translated descriptors such as 'victime' and 'berechtigt' are neutral legal terms, making the co-occurrence results interpretable as legal register rather than social bias; that is a construct-validity caveat about the instrument, not a circular derivation. A validity concern of this kind belongs under correctness risk, not circularity, because the reported associations are empirical facts obtained from independent data and an external descriptor taxonomy.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

Central analysis rests on a translated descriptor instrument, an independence assumption in the binomial test, and an attribution-to-influence assumption. The translation assumption is explicitly flagged by the authors as a limitation.

free parameters (3)
  • Significance threshold alpha = 0.1
    Chosen in Section 3.5.3; unusually permissive and used without multiple-comparison correction, inflating the list of flagged descriptors (Table 7).
  • Chunk text size = 300 words (targeting 512 tokens)
    Hand-set in Section 3.3 to satisfy the model's token limit; determines which descriptors survive into chunks and the counts in Tables 5-6.
  • Summary sentence range = 3 to 26 sentences, trimmed to <=512 tokens
    Ad hoc workaround in Section 3.3; affects descriptor retention in the summarization representation.
assumptions (3)
  • domain assumption English dispreferred descriptors retain their dispreferred social meaning after DeepL/Google translation into German, French, and Italian legal text.
    Section 3.2 constructs the descriptor lists via automatic translation and manual expansion; Section 4.3 concedes this may fail. If false, flagged tokens are legal-register artifacts.
  • domain assumption Each descriptor occurrence is an independent Bernoulli trial with a constant probability of the outcome label.
    Section 3.5 uses the binomial formula; the authors count descriptors rather than facts and note most facts contain up to two descriptors, so occurrences within the same fact are not independent.
  • domain assumption Attention attributions from transformers-interpret indicate the words that drive the model's classification.
    Section 3.6 uses word attributions to infer influence; no control or causal validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analyzing Bias in Swiss Federal Supreme Court Judgments Using Facebook's Holistic Bias Dataset: Implications for Language Model Training." pith.science (2026). https://pith.science/paper/I35URBTH

@misc{pith2026250103324,
  author       = {Pith},
  title        = {Pith review of: Analyzing Bias in Swiss Federal Supreme Court Judgments Using Facebook's Holistic Bias Dataset: Implications for Language Model Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I35URBTH}},
  note         = {Machine review of arXiv:2501.03324}
}
read the original abstract

Natural Language Processing (NLP) is vital for computers to process and respond accurately to human language. However, biases in training data can introduce unfairness, especially in predicting legal judgment. This study focuses on analyzing biases within the Swiss Judgment Prediction Dataset (SJP-Dataset). Our aim is to ensure unbiased factual descriptions essential for fair decision making by NLP models in legal contexts. We analyze the dataset using social bias descriptors from the Holistic Bias dataset and employ advanced NLP techniques, including attention visualization, to explore the impact of dispreferred descriptors on model predictions. The study identifies biases and examines their influence on model behavior. Challenges include dataset imbalance and token limits affecting model performance.

Figures

Figures reproduced from arXiv: 2501.03324 by the authors.

Figure 1
Figure 1. Holistic Bias Dataset - Version 1.1 with respectively “dispreferred” labeled descriptors. 3.1. Selecting Bias-Descriptors For the bias analysis in the facts of court rulings in the Swiss Judgement Prediction Dataset (SJP Dataset) [19], the descriptors labeled as “dispreferred” from the Holistic Bias Dataset [17] are used. The Holistic Bias Dataset includes two versions. The original Version 1.0 contains 620 unique d… view at source ↗
Figure 2
Figure 2. Occurrences of Derived Descriptors in German Training Data, by Instance Label. Gender forms are intentionally used to detect potential double biases. Since there was the translation step and additional similar descriptors have been derived from the descriptors originally labeled as “dispreferred”, the term “derived dispreferred” descriptors is used instead of “labeled” in the following [PITH_FULL_IMAGE:figures/full… view at source ↗
Figure 3
Figure 3. Results of the binomial significance test for chunked data. In the plot for the label “approved” the biased descriptor “en danger” occurs most frequently with a total count of 695, while for “dismissal” the biased descriptor “victime” has a potential influence due to 3928 occurences. The biased labels of the derived descriptors are listed in [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Classification Performance on German Test Data per Contained Descriptor. “berechtigt” (entitled), “Opfer” (victim), “bedroht” (threatened), “victime” (victim), “intitulé” (en￾titled), and “menacé” (threatened) can have a significant influence on the model’s classificat…
Figure 5
Figure 5. Figure 5: Classification Performance on French Test Data per Contained Descriptor. 0 10 20 30 40 # Occurrences of Derived Descriptors in 154 Predicted Chunks. vittima idoneo autorizzata casalinga minacciato legittimo autorizzato autorizzati idonea abilitate legittima in pericolo…
Figure 6
Figure 6. Figure 6: Classification Performance on Italian Test Data per Contained Descriptor. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 8 canonical work pages

  1. [1]

    Identifying biases in legal data: An algorithmic fairness perspective

    J. Sargent, M. Weber, Identifying biases in legal data: An algorithmic fairness perspective, CoRR abs/2109.09946 (2021). URL: https://arxiv.org/abs/2109.09946. arXiv:2109.09946

    work page Pith review arXiv 2021

  2. [2]

    E. Ash, D. L. Chen, A. Ornaghi, Gender attitudes in the judiciary: Evidence from us circuit courts, American Economic Journal: Applied Economics 16 (2024) 314–350

    work page 2024

  3. [3]

    Bhowmick, P

    A. Bhowmick, P. Novosad, S. Asher, E. Ash, B. Siddiqi, C. Goessman, D. Chen, T. Devi, In- group bias in the indian judiciary: Evidence from 5.5 million criminal cases, in: COMPASS ’21: ACM SIGCAS Conference on Computing and Sustainable Societies, Virtual Event, Australia, 28 June 2021 - 2 July 2021, ACM, 2021, p. 47. URL: https://doi.org/10.1145/3460112. ...

    arXiv 2021

  4. [4]

    Bolukbasi, K

    T. Bolukbasi, K. Chang, J. Y. Zou, V. Saligrama, A. T. Kalai, Man is to computer programmer as woman is to homemaker? debiasing word embeddings, in: D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, R. Garnett (Eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016,...

    work page 2016

  5. [5]

    A. C. Islam, J. J. Bryson, A. Narayanan, Semantics derived automatically from language corpora necessarily contain human biases, CoRR abs/1608.07187 (2016). URL: http://arxiv. org/abs/1608.07187. arXiv:1608.07187

    arXiv 2016

  6. [6]

    Gonen, Y

    H. Gonen, Y. Goldberg, Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis,...

    2019

  7. [7]

    Sevim, F

    N. Sevim, F. Sahinuç, A. Koç, Gender bias in legal corpora and debiasing it, Nat. Lang. Eng. 29 (2023) 449–482. URL: https://doi.org/10.1017/S1351324922000122. doi:10.1017/ S1351324922000122

    doi:10.1017/s1351324922000122 2023

  8. [8]

    Gumusel, V

    E. Gumusel, V. Q. Malic, D. R. Donaldson, K. D. Ashley, X. Liu, An annotation schema for the detection of social bias in legal text corpora, in: M. Smits (Ed.), Information for a Better World: Shaping the Global Future - 17th International Conference, iConference 2022, Virtual Event, February 28 - March 4, 2022, Proceedings, Part I, volume 13192 of Lectur...

    work page doi:10.1007/978-3-030-96957-8 2022

Show all 20 references
  1. [9]

    Y. C. Tan, L. E. Celis, Assessing social and intersectional biases in contextualized word representations, in: H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32: Annual Conference on...

    2019

  2. [10]

    Rudinger, J

    R. Rudinger, J. Naradowsky, B. Leonard, B. V. Durme, Gender bias in coreference resolution, in: M. A. Walker, H. Ji, A. Stent (Eds.), Proceedings of the 2018 Conference of the North 15 Sabine Wehnert et al. LIRAI’24 Workshop 1–17 American Chapter of the Association for Computa...

    2018 doi

  3. [11]

    Dinan, A

    E. Dinan, A. Fan, A. Williams, J. Urbanek, D. Kiela, J. Weston, Queens are powerful too: Mitigating gender bias in dialogue generation, in: B. Webber, T. Cohn, Y. He, Y. Liu (Eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2...

    2020 doi

  4. [12]

    J. Zhao, T. Wang, M. Yatskar, R. Cotterell, V. Ordonez, K. Chang, Gender bias in contextual- ized word embeddings, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis- tics:...

    2019 doi

  5. [13]

    R. Qian, C. Ross, J. Fernandes, E. M. Smith, D. Kiela, A. Williams, Perturbation augmen- tation for fairer NLP, in: Y. Goldberg, Z. Kozareva, Y. Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United ...

    2022 doi

  6. [14]

    Wehnert, V

    S. Wehnert, V. Murugadas, P. V. Naik, E. W. D. Luca, Improving robustness in language models for legal textual entailment through artifact-aware training, in: T. Suzumura, M. Bono (Eds.), New Frontiers in Artificial Intelligence - JSAI International Symposium on Artificial Int...

    2024 doi

  7. [15]

    C. May, A. Wang, S. Bordia, S. R. Bowman, R. Rudinger, On measuring social biases in sentence encoders, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Lang...

    2019 doi

  8. [16]

    Nadeem, A

    M. Nadeem, A. Bethke, S. Reddy, Stereoset: Measuring stereotypical bias in pretrained language models, in: C. Zong, F. Xia, W. Li, R. Navigli (Eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer...

    2021 doi

  9. [17]

    i’m sorry to hear that

    E. M. Smith, M. Hall, M. Kambadur, E. Presani, A. Williams, "i’m sorry to hear that": Finding new biases in language models with a holistic descriptor dataset, in: Y. Goldberg, Z. Kozareva, Y. Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in 16 Sabine W...

    2022

  10. [18]

    Wehnert, D

    S. Wehnert, D. Picca, E. W. D. Luca, Mining sentiment and subjectivity in swiss case law, in: S. Wehnert, M. Fiorelli, D. Picca, E. W. D. Luca, A. Stellato (Eds.), Proceedings of the 1st Legal Information Retrieval meets Artificial Intelligence Workshop LIRAI 2023 co-located w...

    2023

  11. [19]

    Niklaus, I

    J. Niklaus, I. Chalkidis, M. Stürmer, Swiss-judgment-prediction: A multilingual le- gal judgment prediction benchmark, in: N. Aletras, I. Androutsopoulos, L. Barrett, C. Goanta, D. Preotiuc-Pietro (Eds.), Proceedings of the Natural Legal Language Pro- cessing Workshop 2021, NL...

    2021

  12. [20]

    Niklaus, V

    J. Niklaus, V. Matoshi, M. Stürmer, I. Chalkidis, D. E. Ho, Multilegalpile: A 689gb multilin- gual legal corpus, CoRR abs/2306.02069 (2023). URL: https://doi.org/10.48550/arXiv.2306. 02069. doi:10.48550/ARXIV.2306.02069. arXiv:2306.02069. 17

    2023 doi arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.