Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech

Florian Ludwig; Frederike Zufall; Torsten Zesch

REVIEW 5 major objections 6 minor 3 references

Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs still lag legal experts on punishable hate speech, at every conditioning level

desk verdict A solid negative result on legal conditioning for punishable hate speech, but the central claim rests on 150 synthetic samples and no significance testing, so the ranking of approaches is provisional. read the letter →

arxiv 2506.03009 v1 pith:KYY3737A submitted 2025-06-03 cs.CL cs.AI

Florian Ludwig , Torsten Zesch , Frederike Zufall This is my paper

classification cs.CLcs.AI

keywords hatespeechdetectionlargelanguagemodelslegalconditioningGermancriminalcodeincitementtohatredpromptengineeringabstractionlevels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can be conditioned on a legal system by feeding them legal knowledge at different levels of abstraction — constitutional rights, statutory text, and case-law definitions and examples. Using the German offense of incitement to hatred (§ 130(1) StGB) as the test case, it compares several prompting approaches across four models. The central finding is that no level of abstraction closes the gap between model judgments and legal experts' annotations. Models given only abstract knowledge often contradict themselves or hallucinate, while models given concrete definitions can identify protected target groups but misclassify the specific criminal conduct.

What carries the argument

The central mechanism is the operationalization of a legal system as three abstraction levels — constitutional law, statutory law, and case law — together with the 'stages' decomposition that splits a § 130(1) assessment into protected target-group identification and target-conduct classification. This decomposition turns a single classification question into prompt chains, and it is also the yardstick by which the paper measures whether more concrete knowledge (definitions, examples) improves performance. The hallucination probe, which prompts with fabricated paragraph numbers, is the diagnostic that reveals whether a model is actually reading the legal text or ignoring it.

What would settle it

If a prompting or fine-tuning setup reaches expert-level agreement (Cohen's κ at or above the expert agreement of about 0.79) on a larger corpus of real social media posts with court or multi-expert gold labels, the claim that a significant performance gap persists regardless of conditioning level is refuted. Alternatively, if a model prompted with a fabricated paragraph number (e.g. § 400 StGB) produces near-zero agreement with expert labels on the same 150 samples, the paper's hallucination evidence is contradicted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that LLMs do not internalize the structure of a legal system deeply enough to replicate expert legal assessment. No matter whether the prompt supplies only a statute's title, the full statutory text, a simplified text, a staged decomposition of the offense, or case-law definitions and examples, agreement with legal experts remains far below expert-expert agreement. Abstract title conditioning produces logically contradictory judgments in up to 60% of cases, and models prompted with non-existent paragraph numbers still produce labels as if the paragraph were relevant, indicating hallucination rather than reasoning. Concrete conditioning helps models identify the protected target groups, but they struggle to classify the target conduct that makes speech punishable under § 130(1).

Load-bearing premise

The evaluation assumes that the 150 manually crafted posts annotated by legal experts are a sufficient gold standard for punishable hate speech under § 130(1) StGB; if those labels or the annotation framework do not represent how courts actually decide, the measured gap and the ranking of approaches would not transfer to real posts.

Editorial extensions

If this is right

Adding case-law definitions to the stages decomposition improves agreement for most models over plain statutory stages, but adding examples has mixed effects and can trigger safety refusals.
Models extract target groups from text better than they classify them into predefined categories, suggesting that generation-style outputs are the stronger mode for this subtask.
Prompting with only a statute's title can outperform full statutory text, but the hallucination probe shows this is because the model ignores the specific paragraph; performance is near-random yet above chance.
The two-step free-text-then-JSON inference protocol avoids the degradation caused by forcing structured output, providing a reproducible evaluation recipe for legal classification tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the gap is robust, LLM-based hate-speech moderation for German law should not make final legal determinations; a human-in-the-loop system that uses the model for target-group extraction and a rule-based or expert check for target conduct would be a safer design.
The contradiction measure between freedom-of-speech and punishability judgments is a cheap consistency diagnostic that could be reused in any legal classification task, and it could be sharpened by testing whether instructing the model to restate the legal elements first reduces the contradiction rate.
The 150-sample synthetic dataset may underestimate real-world difficulty; running the same prompts on real social media posts with court-adjudicated labels is a direct testable extension that would either strengthen or qualify the paper's conclusion.
The fabricated-paragraph probe shows that title-only prompting can produce labels for non-existent statutes; adapting this probe as a standard sanity check would help separate genuine legal reasoning from pattern-matching in future legal NLP systems.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper's central finding is credible: none of the tested conditioning approaches—from fundamental-rights titles to statutory text to case-law definitions and examples—gets current LLMs anywhere near expert-level agreement on German §130(1) hate speech detection. Best model kappa is around 0.39 against 0.79 for experts, and the control with non-existent or irrelevant paragraphs is a genuinely good idea: it shows that title-only prompting leads models to hallucinate legal relevance, while adding statutory text reduces that behavior. The abstraction-level framing is a useful way to organize prompt designs, even if the individual techniques (title/text prompting, CoT, decomposition, in-context examples) are not new. The paper reports results honestly and does not overclaim; the conclusion that a gap persists regardless of conditioning follows from the data.

The soft spots are real but do not overturn the main conclusion. The evaluation uses only the 150 manually crafted samples from Zufall et al. (2022), explicitly excluding the 850 real-world posts. That is a serious limitation: the gap may be partly an artifact of a small, synthetic test set with unnatural phrasing and unbalanced classes. The stress-test note is correct that the Limitations section does not mention this, and it should. The Stages prompts are built from the same annotation framework that produced the gold labels, so the Stages condition is partly benchmark-relative. The paper also reports no confidence intervals or significance tests; some across-condition differences (e.g., examples helping one model and hurting another) are likely noise and should not be overinterpreted. No code or full prompt paraphrases are released, which makes replication harder, though the prompt examples in the appendix help.

For all that, the core negative result is robust in magnitude—the gap to experts is too large to be explained away by the small test set alone. This is a fair empirical contribution to the legal-LLM and hate-speech literatures. It deserves a serious referee, but the revision should address the synthetic-only evaluation, add variance reporting or significance testing, and release the prompts and code. I would not cite it as a definitive statement about real-world hate speech, but I would cite the hallucination control.

My recommendation: send it to peer review, with the expectation of major revision on the evaluation robustness.

Referee Report

5 major / 6 minor

Summary. This paper proposes and evaluates a hierarchy of prompting strategies for LLMs on the task of detecting punishable hate speech under German Criminal Code §130(1) StGB, conditioning models at different levels of abstraction of the legal system: constitutional fundamental rights, statutory law, and case law. The authors test four LLMs (Phi-3, Llama 3.1, Command R+, Qwen 2) on 150 manually crafted samples from Zufall et al. (2022), with expert annotations as gold standard. The central finding is that none of the conditioning approaches closes the gap between model performance and legal experts: the best model Kappa is around 0.41, far below the expert agreement of 0.79. A control experiment using nonexistent or irrelevant paragraphs shows that models prompted with abstract legal titles often hallucinate legal reasoning. The paper also decomposes the task into target-group and target-conduct subtasks, finding that target-group extraction works better than target-conduct classification.

Significance. The central negative result—that no tested prompting approach allows current LLMs to approach expert-level legal judgment on this task—is a useful and credible finding for the legal NLP community. The paper's operationalization of legal-system abstraction levels is a thoughtful conceptual contribution, and the nonexistent-paragraph control in Table 3 is a strong, reusable method for testing whether models actually use the legal norm provided in the prompt. The fine-grained analysis of target groups versus target conduct also offers diagnostic value. However, the evaluation is restricted to a small synthetic dataset, and several secondary claims rest on analyses that need correction or statistical support; if these are addressed, the paper's main message would be substantially more robust.

major comments (5)

[Section 3.1] The evaluation uses only the 150 manually crafted samples from Zufall et al. (2022) and explicitly excludes the 850 real-world posts. The abstract and conclusion state a general 'significant performance gap' without qualifying that it is demonstrated only on this small synthetic set. Because the samples were manually constructed to resemble hate speech, they may not reflect the distribution of actual social-media posts, so the measured gap may not transfer to real-world data. Please qualify the central claim or provide additional evidence, for instance by re-annotating a sample of the real-world posts or analyzing distributional similarity.
[Section 4.1] The definition of a 'contradictory decision' includes statements classified as 'neither covered by freedom of expression nor punishable under §130(1)'. This pair is not logically contradictory: a statement can be outside the scope of Art. 5(1) GG without being punishable under §130(1), for example a purely factual assertion. Consequently, the reported contradiction rates (60% for Cmd title-based, 34% for text-based; 27% and 38% for Qwen) are inflated and do not establish self-contradiction. Please recompute the rates using only the 'both covered and punishable' case, or provide a legal justification for treating the 'neither' pair as contradictory.
[Tables 2, 4, and 5] The Kappa values are reported without confidence intervals or significance tests. With six prompt paraphrases per condition and 150 items, observed differences between approaches—for example the claim that Statutory Title Conditioning outperformed the other approaches in three of four cases (Section 4.2)—may be within sampling noise. Please provide bootstrap confidence intervals for the main Kappa values, and use a paired test when claiming relative improvements between conditions.
[Section 4.2 / Conclusion] The claim that 'approaches with more concrete legal knowledge performed worse on average than high-level end-to-end classification approaches' is not clearly supported by Table 2. Averaging the four models gives Statutory Title .27, Definitions .24, and Definitions & Examples .22; these differences are small and no statistical test is reported. Please either support this claim statistically or soften it to reflect the actual pattern.
[Sections 2.2 and 3.1] The Stages prompts are constructed directly from the annotation framework of Zufall et al. (2022), which is also the source of the gold labels. This makes the Stages condition partly benchmark-relative, since the same legal decomposition is baked into both the prompt and the ground truth. While the overall low performance of Stages mitigates the concern, the paper should explicitly acknowledge this potential circularity in the Limitations section.

minor comments (6)

[Abstract / Section 1] The sentence 'This paper examines different approaches to conditioning LLMs at multiple levels of abstraction...' appears verbatim in both the abstract and the introduction; please remove the duplication.
[Table 3] Several cells in Table 3 are marked with '/'; please clarify whether those experiments were not run or produced no interpretable output.
[Tables 1 and 2] Table 1 reports F1 scores for human annotators and baselines, but Table 2 reports only Kappa for the LLMs; adding F1 would enable a direct comparison across tables.
[Section 3.1] The phrase 'the dataset consists of 1,000 German instances, whereby 850 samples are real world samples that have not been published' is ambiguous; it should be clarified that the 850 real-world samples are not publicly released, while the 150 manually crafted ones are.
[Appendix C] The prompt example labeled 'Statutory Law: Stages + Deﬁnitions' appears to be part of the Case Law section; please check the internal consistency of the appendix headings.
[Section 4.1] The sentence 'Adding legal text did not improve the performances in three out of four cases' should specify which cases are meant, since Table 2 shows mixed effects across models.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central performance-gap claim is tested against an external expert-labeled dataset, and the self-cited Zufall et al. framework is a conditioning input, not a fitted prediction.

full rationale

The paper's central claim is an empirical one: across several prompting approaches, LLMs still underperform legal experts on the Zufall et al. hate-speech dataset. No model parameter is fitted to the expert labels; the SVM baseline is the only trained method and is used only as a comparison point. The Stages prompts are deliberately constructed from the Zufall et al. annotation framework, and the paper reports that this conditioning does not close the gap to experts, so the framework is not being 'predicted back' as a success. The self-citation to Zufall et al. (2022) supplies a public, expert-annotated evaluation set and the operationalized legal definitions; it does not by itself establish the paper's conclusion, which rests on the actual LLM experiments. The concern that the evaluation uses only the 150 manually crafted samples and excludes the 850 real-world posts is an external-validity limitation, and the Limitations section's silence on this point is a reporting gap, not a circularity. No equation or prediction is shown to reduce to its own inputs by construction, and no load-bearing argument rests on an unverified self-citation alone.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on domain assumptions about the legal annotation framework and the representativeness of the test set, not on fitted mathematical parameters or new entities. No numerical parameters are fit to data.

assumptions (4)

domain assumption The annotation framework in Zufall et al. (2022) correctly operationalizes Section 130(1) StGB for NLP classification.
Used as the gold standard for all 150 test instances and as the basis for the Stages, Definitions, and Examples prompts (Sections 3.1 and 2.2).
domain assumption The three abstraction levels (constitutional law, statutory law, case law) are separable and can be mapped to distinct prompt conditions.
This is the paper's key modeling assumption, introduced in Section 2 and Figure 1; if the levels are not independent or the prompts do not isolate them, the comparison is invalid.
domain assumption Cohen's Kappa between model predictions and expert annotations is a valid measure of legal assessment performance.
Used throughout Section 4 as the primary metric; it treats experts as ground truth and ignores the possibility of expert disagreement beyond the reported agreement.
domain assumption The 150 manually crafted samples are representative enough to compare conditioning approaches.
Section 3.1 notes the 850 real-world unpublished samples are excluded; the paper generalizes from the 150 synthetic posts.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech." pith.science (2026). https://pith.science/paper/KYY3737A

@misc{pith2026250603009,
  author       = {Pith},
  title        = {Pith review of: Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KYY3737A}},
  note         = {Machine review of arXiv:2506.03009}
}

read the original abstract

The assessment of legal problems requires the consideration of a specific legal system and its levels of abstraction, from constitutional law to statutory law to case law. The extent to which Large Language Models (LLMs) internalize such legal systems is unknown. In this paper, we propose and investigate different approaches to condition LLMs at different levels of abstraction in legal systems. This paper examines different approaches to conditioning LLMs at multiple levels of abstraction in legal systems to detect potentially punishable hate speech. We focus on the task of classifying whether a specific social media posts falls under the criminal offense of incitement to hatred as prescribed by the German Criminal Code. The results show that there is still a significant performance gap between models and legal experts in the legal assessment of hate speech, regardless of the level of abstraction with which the models were conditioned. Our analysis revealed, that models conditioned on abstract legal knowledge lacked deep task understanding, often contradicting themselves and hallucinating answers, while models using concrete legal knowledge performed reasonably well in identifying relevant target groups, but struggled with classifying target conducts.

Figures

Figures reproduced from arXiv: 2506.03009 by the authors.

**Figure 1.** Dierent levels of abstraction in a legal system and their operationalization. et al., 2024). The question arises with respect to numerous legal problems that may be put forward to generative models in the form of prompts. For traditional supervised machine learning approaches, legal knowledge is implicitly encoded in the annotated data instances. However, these approaches must first reconstruct the actual task fr… view at source ↗

**Figure 2.** Approaches to legal knowledge conditioning, which were investigated in our experiments. be posed on several levels of abstraction starting from the constitutional level by referring to a protection by fundamental rights (Sec 2.1). Furthermore, the problem may be addressed at the level of statutory law, namely the respective provision of the German Criminal Act, § 130(1) (Sec 2.2). Even further down in the legal sy… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

3 extracted references · 2 canonical work pages

[1]

incites hatred against a national, racial, religious group or a group deﬁned by their ethnic origin, against sections of the population or individuals on account of their belonging to one of the aforementioned groups or sections of the population, or calls for violent or arbitrary measures against them or

work page
[2]

C Examplary Prompts In this section, we provide example prompts for the dierent approaches, corresponding to the dierent levels of abstractions of the legal system

violates the human dignity of others by insulting, maliciously maligning or defaming one of the aforementioned groups, sections of the population or individuals on account of their belonging to one of the aforementioned groups or sections of the population incurs a penalty of imprisonment for a term of between three months and ﬁve years. C Examplary Promp...

work page
[2021]

Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho

Detecting hate speech with gpt-3.arXiv preprint arXiv:2103.12407. Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024a. Large legal ﬁctions: Proﬁl- ing legal hallucinations in large language models. Journal of Legal Analysis, 16(1):64–93. Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024b.Large Legal Fictions: Pro- ﬁling Legal Ha...

arXiv 2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

incites hatred against a national, racial, religious group or a group deﬁned by their ethnic origin, against sections of the population or individuals on account of their belonging to one of the aforementioned groups or sections of the population, or calls for violent or arbitrary measures against them or

work page

[2] [2]

C Examplary Prompts In this section, we provide example prompts for the dierent approaches, corresponding to the dierent levels of abstractions of the legal system

violates the human dignity of others by insulting, maliciously maligning or defaming one of the aforementioned groups, sections of the population or individuals on account of their belonging to one of the aforementioned groups or sections of the population incurs a penalty of imprisonment for a term of between three months and ﬁve years. C Examplary Promp...

work page

[3] [2021]

Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho

Detecting hate speech with gpt-3.arXiv preprint arXiv:2103.12407. Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024a. Large legal ﬁctions: Proﬁl- ing legal hallucinations in large language models. Journal of Legal Analysis, 16(1):64–93. Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024b.Large Legal Fictions: Pro- ﬁling Legal Ha...

arXiv 2023