Pith. sign in

REVIEW 4 major objections 6 minor 24 references

Dense retrieval improves French immigration permit recommendations over parametric LLMs alone, especially on permit type.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 14:37 UTC pith:XOR4FMPQ

load-bearing objection Useful first public benchmark and controlled RAG baseline for French immigration admin guidance; the headline gains are directionally credible but partly undercut by small n and annotation seeded by the same RAG/Qwen stack. the 4 major comments →

arxiv 2607.24449 v1 pith:XOR4FMPQ submitted 2026-07-27 cs.IR cs.AI

Evaluating RAG for French immigration law: a benchmark and baseline study

Annia Abtout , Julien Delaunay , Monika Ewa Rakoczy This is my paper
classification cs.IR cs.AI
keywords RAGhuman resourcesimmigration lawFrench administrative guidancelegal benchmarkpermit recommendationdense retrievalcitation coverage
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

HR teams hiring foreign workers in France must pick the right residence permit, list the right documents, and ground the advice in layered law (CESEDA, labour rules, bilateral agreements, Service-Public). Existing legal AI benchmarks do not cover this administrative guidance task. This paper releases a public benchmark of 52 annotated synthetic applicant profiles and compares a plain LLM against the same model with dense retrieval over a curated legal corpus, at two sizes (Qwen3.5-9B and 27B). Retrieval raises permit-type accuracy and full permit match at both scales, and also helps document lists and legal citations when the permit is already correct. The authors conclude that grounding in retrieved law is needed for more reliable guidance in this domain, and that hybrid retrieval is a natural next step.

Core claim

On 52 annotated synthetic profiles, dense RAG improves administrative guidance versus a parametric baseline at both model scales, most clearly on permit-type accuracy (50% to 75% for 27B; 30.8% to 57.7% for 9B) and full permit match (25% to 34.6%; 9.6% to 25%), with further gains on documents and citation coverage when evaluation is restricted to correct permits. Retrieval mainly reduces confusion between legal regimes; residual errors and incomplete citation coverage remain.

What carries the argument

A three-output benchmark (permit recommendation with four attributes, required documents, legal references) plus a controlled Dense RAG setup: the same prompt and legal corpus, top-k dense chunks (BGE-M3) injected versus LLM-only, evaluated with component match, document F1 under full-permit filter, and recall-oriented citation coverage.

Load-bearing premise

The load-bearing premise is that 52 rule-simulated synthetic profiles, annotated with help from a same-family RAG model whose answers annotators corrected, are a fair enough stand-in for real international-recruitment cases that the measured RAG gains will generalize.

What would settle it

Run the same LLM-only versus Dense RAG comparison on a larger set of real (or independently annotated) French international-recruitment cases outside the Qwen family and check whether permit-type and full-permit gains of similar size still appear.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Public evaluation of French immigration HR guidance can now use a shared multi-output benchmark instead of ad hoc cases.
  • Permit-family and legal-regime errors shrink when dense legal context is added, at both 9B and 27B scale.
  • Document and citation quality still lag and stay partly scale-dependent, so retrieval alone does not finish the job.
  • The authors’ own next system step is hybrid retrieval rather than larger parametric models alone.
  • A planned second annotation pass toward ~200 profiles would tighten claims about downstream document and citation metrics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Civil-law administrative procedures with hierarchy (bilateral agreements over CESEDA) may need explicit source-priority prompting plus retrieval, not memorization alone.
  • Conditioning document and citation scores on full permit match will keep those metrics fragile until permit accuracy is high or evaluation is decoupled.
  • Annotation sycophancy toward model-suggested documents and fees is a structural risk for any legal benchmark built with LLM draft labels.
  • The same protocol could transfer to other national immigration or work-permit regimes that mix statute, bilateral deals, and procedural portals.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces a publicly released benchmark for French immigration administrative guidance in the context of international recruitment: 52 synthetic applicant profiles annotated for (i) a four-attribute residence permit recommendation, (ii) required supporting documents, and (iii) legal citations, with annotation guidelines, inter-annotator agreement analysis, and a curated legal corpus (CESEDA, Employment Act, bilateral agreements, Service-Public). Using this benchmark, the authors run a controlled comparison at two Qwen3.5 scales (9B, 27B) between an LLM-only condition and a dense RAG condition (BGE-M3 embeddings over the same corpus, identical prompt). Headline results: permit_type accuracy improves from 50.0% to 75.0% (27B) and 30.8% to 57.7% (9B), with parallel gains in full permit match, document F1, and citation coverage, the latter two evaluated only on full-permit-match subsets. The authors conclude that retrieval grounding materially improves administrative guidance reliability in this domain.

Significance. If the results hold, the paper makes two contributions of value. First, it fills a genuine gap: existing legal benchmarks (LegalBench, LawBench, LexEval, LexRAG) do not cover French administrative immigration guidance, and the multi-output formulation (decision + procedure + legal grounding) is a faithful model of the HR use case. Second, the work is unusually transparent for a first benchmark study: code, benchmark, annotation guidelines, generation parameters, IAA analysis, error taxonomies, and metric conditioning rules are all released or documented, and the conditioning choices (documents/citations evaluated only on full permit match; citation coverage as a lower bound) are stated plainly rather than buried. The two-scale controlled comparison (same prompt, same corpus, retrieval on/off) is a clean design for the directional claim. These are real strengths. The significance is tempered by the scale of the evidence (n=52, single model family, conditional subsets of n=5–18) and by an annotation-provenance concern detailed below that bears directly on the headline numbers; both are addressable within the manuscript's scope.

major comments (4)
  1. [§3.3, Supp. A.4, Table 1] §3.3 and Table 1: the gold standard was seeded by the same architecture being evaluated. Candidate recommendations were generated by 'a RAG-augmented Qwen3.5-27B over a curated legal corpus' and then validated or corrected by annotators. Where annotators accepted the candidate unchanged, the gold label is structurally closer to the Dense RAG condition (same model family, same corpus, same retrieval) than to the LLM-alone condition. Supp. A.4 explicitly documents that this happened ('a bias of sycophancy... permitted some document not required to be selected... or incorrect fees'). This is not necessarily fatal — annotators worked against authoritative sources and disagreements were adjudicated — but the size of the effect is currently unquantified, and with a 13-profile swing driving the headline permit_type result it is load-bearing. Required revisions: (a) report how many of the 52 gol
  2. [§5, Tables 1, 5, 7] No uncertainty quantification accompanies any of the headline numbers despite n=52. The permit_type gain at 27B (+25.0 pts) is 13 profiles; whether this is statistically meaningful depends entirely on the discordant-pair structure of the paired comparison, which is not reported. A McNemar test (or exact binomial on discordant pairs) and bootstrap confidence intervals for all Table 1 deltas are cheap to compute on paired per-profile outputs and should be added. The same applies to Table 5 (document F1) and Table 7 (citation coverage), where the conditional subsets are n=5–18 and differences of 20+ points may well be within noise. The authors repeatedly say results 'should be read qualitatively/indicative only'; with per-profile outputs available in the released code, formal intervals are preferable to disclaimers.
  3. [§4.2, Tables 5 and 7] The document and citation metrics (Tables 5 and 7) are computed on condition-specific full_permit_match subsets that differ across conditions: e.g., Qwen-27B document F1 compares 9 profiles (LLM-alone) against 12 profiles (Dense RAG), and these are not the same profiles. Document F1 and citation coverage therefore confound system quality with subset composition — the RAG condition is evaluated on a partly different, potentially easier or harder, set of profiles. At minimum, report paired metrics on the intersection of profiles correct under both conditions, or report an end-to-end (unconditional) document/citation metric over all 52 profiles alongside the conditional one. As it stands, the cross-condition deltas in Tables 5 and 7 are not well-defined comparisons.
  4. [§4.1, Supp. C.1, Fig. 8] The LLM-alone condition is described as 'a parametric baseline' (§4.1), but the shared prompt (Supp. C.1, Fig. 8) hard-codes precisely the legal special cases that the error analysis identifies as the main failure mode: Franco-Algerian Agreement precedence over CESEDA, EU/EEA free-movement rights, and shortage-occupation effects (cf. Table 4, 'legal regime confusion'). Both conditions receive this, so the retrieval on/off attribution is internally valid; however, the baseline is not measuring what the model knows parametrically — it is measuring the model plus substantial hand-encoded legal knowledge. This matters for the paper's framing ('retrieval partly compensates for limited parametric knowledge', §6) and for comparability with other work: the text should state explicitly that the baseline includes engineered legal rules, and ideally include an ablation with a minimal prompt to sepa
minor comments (6)
  1. [Table 1] Table 1: is_renewal decreases for Qwen-27B under Dense RAG (88.5%→82.7%, −5.8 pts) while all other components improve. This regression is not discussed anywhere; a sentence on its cause (and how many profiles it represents) is warranted.
  2. [Table 1] Table 1, mention* row: the conditioning set differs per condition (profiles with correct permit_type), so the mention* comparison has the same subset-composition issue as Tables 5/7 in miniature. State the subset sizes per cell.
  3. [§4.1] §4.1: the value of top-k for dense retrieval is never stated in the main text or supplement (only 'top-k most relevant chunks'). For reproducibility of the central comparison, give k, the chunking size, and any reranking. The BGE-M3 matching threshold (0.80) is documented in Supp. B.2, but the retrieval-side k appears nowhere.
  4. [§4.2, Table 1] §4.1/§5: visa_required accuracy is ~54% in all four conditions — essentially uninformative. Since full_permit_match requires all four components, the visa field acts as a near-random gate on the headline full-match metric. Consider either motivating the field's inclusion, or reporting full match excluding visa_required as a secondary metric, given the prompt admittedly does not emphasise visa requirements.
  5. [General] Presentation: several typographical issues, e.g. 'there is no publicly available benchmarks' (§1), 'recommandation' (Supp. A.4), 'repercutates' (Supp. A.5), and unclosed parentheses in the captions of Figures 3 and 4. Also 'MediaTech' is introduced with a footnote but its role relative to the authors' own pipeline (pre-computed embeddings? hosting?) could be one clause clearer.
  6. [§2, References] Reference [8] ('All for law and law for all') is cited to support the claim that models 'exhibit sycophancy bias, accepting erroneous legal premises'; from its title and venue (arXiv preprint on adaptive RAG pipelines) this does not appear to be the right source for that claim — [17] (Sharma et al.) is. Please check the citation.

Circularity Check

1 steps flagged

Mild evaluation contamination: gold candidates were seeded by RAG-Qwen3.5-27B (same family/architecture later scored), but labels were human-adjudicated—not definitional circularity.

specific steps
  1. other [§3.3 Dataset construction; also Supp. A.4, §6]
    "To facilitate annotation, we employed a RAG-augmented Qwen3.5-27B [15] over a curated legal corpus (Supp. A.2) to generate candidate recommendations, which annotators then validated or corrected using the INCEpTION annotation tool [9]."

    Gold labels were seeded by the same architecture later evaluated (dense retrieval over the same legal corpus, Qwen3.5-27B family). On profiles where annotators accepted the RAG draft with little change—Supp. A.4 notes LLM suggestions introduced sycophancy bias on documents and fees—the gold is closer by construction to Dense RAG outputs than to LLM-alone, so part of the reported permit_type gain (e.g. 50%→75% at 27B) can reflect agreement with the annotation model rather than independent legal truth. Not fully circular: humans double-annotated and adjudicated; paper flags self-preference in §6. Residual contamination, not definitional identity.

full rationale

This is an empirical benchmark paper, not a first-principles derivation. Permit/document/citation scores are external matches to human gold under fixed normalisation rules, so the central claim (Dense RAG beats LLM-alone on permit_type and full_permit_match) is not true by algebraic construction or by renaming a fitted parameter. The only circularity-adjacent issue is annotation seeding: §3.3 states candidate recommendations were produced by a RAG-augmented Qwen3.5-27B over the same curated corpus, then validated or corrected by annotators; Supp. A.4 and §6 explicitly note sycophancy and same-family self-preference risk. Where annotators accepted RAG drafts unchanged, gold is structurally closer to the Dense RAG condition than to LLM-alone, which can inflate the reported Δ without measuring pure legal correctness. That is evaluation contamination, not self-definitional math. Mitigations present in the paper (double annotation, third-party adjudication, HR expert, shared legal source set, public guidelines) keep the issue moderate-low. No self-citation uniqueness theorem, no fitted-input-called-prediction, and no ansatz smuggling load the result. Score 3 reflects one real but partial contamination path that does not collapse the claim by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The claim rests on standard IR/LLM evaluation assumptions plus domain choices: synthetic INSEE-calibrated profiles stand in for real cases; a filtered multi-source French legal corpus is sufficient; human double annotation with expert adjudication defines gold; dense top-k chunk retrieval is a fair RAG baseline; and match rules (permit normalization, 0.80 embedding threshold, citation canonicalization) measure operational quality. No new physical entities; free choices are engineering thresholds and taxonomy design rather than fitted scientific constants.

free parameters (3)
  • dense_retrieval_top_k = top-k (value not highlighted as tuned in main results)
    Number of chunks injected into the prompt; controls how much legal context the RAG condition sees. Exact k is not stressed as swept in the main text.
  • document_embedding_cosine_threshold = 0.80
    BGE-M3 match threshold chosen after trying 0.65–0.80; 0.80 retained for precision. Directly affects document F1.
  • synthetic_profile_generation_weights = e.g. Africa 42.2%, salary skew-normal α=3 center €35k, 75% permanent employment (Supp. A.1)
    INSEE-inspired nationality, education, salary skew-normal, and permit-duration sampling parameters shape the 52-profile distribution and thus measured accuracies.
axioms (5)
  • domain assumption Dense vector retrieval over a validity-filtered CESEDA, Employment Act, bilateral-agreement, and Service-Public corpus is an adequate operationalization of legal grounding for this task.
    Sections 3.1, 4.1 and Supp. A.2 treat corpus construction and dense RAG as the retrieval condition without proving optimality versus sparse or hybrid retrieval.
  • domain assumption Synthetic applicant profiles calibrated on public INSEE statistics plus controlled legal-regime coverage are valid proxies for evaluating international-recruitment guidance.
    Section 3.3 justifies synthetics via privacy and rare-case coverage; generalization to real cases is assumed for the practical claim.
  • domain assumption Double annotation with third-annotator adjudication against a predefined authoritative source list yields a usable gold standard despite moderate IAA on permits and documents.
    Section 3.3 and Supp. A.4; disagreements on optional EU permits and renewal vs upgrade are acknowledged but gold is still used as ground truth.
  • ad hoc to paper Full permit match may gate document and citation metrics so downstream quality is not confounded by upstream permit errors.
    Section 4.2 evaluation design; produces very small conditional subsets and makes cross-scale document/citation contrasts only indicative.
  • standard math Standard embedding similarity and rule-based normalisation are sufficient to score open-ended document lists and legal citations.
    Supp. B matching protocol; common IR evaluation practice, with extra administrative consistency filters for fees and address recency.
invented entities (2)
  • French immigration administrative multi-output benchmark (52 synthetic profiles) independent evidence
    purpose: Provide shared evaluation of permit recommendation, required documents, and legal citation coverage for international recruitment.
    Core contribution artifact; not a physical entity but a new dataset construct the claims depend on.
  • Closed 16-category permit_type taxonomy with four-attribute full match no independent evidence
    purpose: Decompose permit advice into is_renewal, permit_type, mention, visa_required for component scoring.
    Table 2 / Supp. A.3; evaluation-specific structuring of French permit practice rather than a new legal category in law.

pith-pipeline@v1.2.0-grok45-kimik3 · 16378 in / 3737 out tokens · 63142 ms · 2026-07-31T14:37:28.392043+00:00 · methodology

0 comments
read the original abstract

International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks. We present a publicly available benchmark and first comparative evaluation for this domain, covering permit-type recommendation, required-document retrieval, and legal citation coverage. Comparing a parametric LLM baseline against dense retrieval augmentation at two model scales (Qwen3.5-9B and -27B) on 52 annotated synthetic profiles, we find that retrieval improves administrative guidance at both scales, most notably permit-type accuracy. Our results confirm that retrieval grounding is important for more reliable administrative guidance in this domain, and motivate further investigation of hybrid retrieval strategies.

Figures

Figures reproduced from arXiv: 2607.24449 by Annia Abtout, Julien Delaunay, Monika Ewa Rakoczy.

Figure 1
Figure 1. Figure 1: Pairwise κ on permit-type annotations (training profiles excluded) [PITH_FULL_IMAGE:figures/full_fig_p017_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Confusion matrix for the permit annotation (counts). ‘Renewal’ and visas [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Pairwise F1 on required documents (calibration profiles excluded, and [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Pairwise F1 on legal basis (calibration profiles excluded, and only profiles [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Distribution of recommended permit types (regardless of renewal status) [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of recommended documents per permit type (regardless of [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Legal basis distribution per permit type (regardless of renewal status) [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Role-conditioned user prompt. Tokens in braces are substituted at runtime [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 4 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2303.08774 (2023)

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    Pith/arXiv arXiv 2023

  2. [2]

    In: Ku, L.W., Martins, A., Srikumar, V

    Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., Liu, Z.: M3-embedding: Multi- linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation. In: Ku, L.W., Martins, A., Srikumar, V. (eds.) Findings of the Association for Computational Linguistics: ACL 2024. pp. 2318–2335. As- sociation for Computational Linguisti...

    doi:10.18653/v1/2024.findings-acl.137 2024

  3. [3]

    Educational and psycho- logical measurement20(1), 37–46 (1960)

    Cohen, J.: A coefficient of agreement for nominal scales. Educational and psycho- logical measurement20(1), 37–46 (1960)

    1960

  4. [4]

    Journal of Legal Analysis16(1), 64–93 (2024) 8 A

    Dahl, M., Magesh, V., Suzgun, M., Ho, D.E.: Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis16(1), 64–93 (2024) 8 A. Abtout et al

    2024

  5. [5]

    In: Proceedings of the 2024 conference on empirical methods in natural language processing

    Fei, Z., Shen, X., Zhu, D., Zhou, F., Han, Z., Huang, A., Zhang, S., Chen, K., Yin, Z., Shen, Z., et al.: Lawbench: Benchmarking legal knowledge of large language models. In: Proceedings of the 2024 conference on empirical methods in natural language processing. pp. 7933–7962 (2024)

    2024

  6. [6]

    Advances in neural information processing systems36, 44123–44279 (2023)

    Guha, N., Nyarko, J., Ho, D., Ré, C., Chilton, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D., Zambrano, D., et al.: Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in neural information processing systems36, 44123–44279 (2023)

    2023

  7. [7]

    In: Proceedings of the 1st workshop on customizable nlp: Progress and challenges in customizing nlp for a domain, appli- cation, group, or individual (customnlp4u)

    Kalra, R., Wu, Z., Gulley, A., Hilliard, A., Guan, X., Koshiyama, A., Treleaven, P.C.: Hypa-rag: A hybrid parameter adaptive retrieval-augmented generation sys- tem for ai legal and policy applications. In: Proceedings of the 1st workshop on customizable nlp: Progress and challenges in customizing nlp for a domain, appli- cation, group, or individual (cus...

    2024

  8. [8]

    arXiv preprint arXiv:2508.13107 (2025)

    Keisha, F., Singh, P., Fernandes, D., Manivannan, A., Wicaksono, I., Ahmad, F., Rim, W.B., et al.: All for law and law for all: Adaptive rag pipeline for legal research. arXiv preprint arXiv:2508.13107 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    In: Proceedings of the 27th International Conference on Computa- tional Linguistics: System Demonstrations

    Klie, J.C., Bugert, M., Boullosa, B., Eckart de Castilho, R., Gurevych, I.: The INCEpTION platform: Machine-assisted and knowledge-oriented interactive an- notation. In: Proceedings of the 27th International Conference on Computa- tional Linguistics: System Demonstrations. pp. 5–9. Santa Fe, New Mexico (2018), https://www.aclweb.org/anthology/C18-2002

    2018

  10. [10]

    Advances in neural information processing systems 33, 9459–9474 (2020)

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020)

    2020

  11. [11]

    In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Li, H., Chen, Y., YiRan, H., Ai, Q., Chen, J., Yang, X., Yang, J., Wu, Y., Liu, Z., Liu, Y.: Lexrag: Benchmarking retrieval-augmented generation in multi-turn legal consultation conversation. In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 3606–3615 (2025)

    2025

  12. [12]

    Advances in Neural Information Processing Systems37, 25061–25094 (2024)

    Li, H., Chen, Y., Ai, Q., Wu, Y., Zhang, R., Liu, Y.: Lexeval: A comprehensive chinese legal benchmark for evaluating large language models. Advances in Neural Information Processing Systems37, 25061–25094 (2024)

    2024

  13. [13]

    Advances in Neural Information Processing Systems37, 68772– 68802 (2024)

    Panickssery, A., Bowman, S.R., Feng, S.: Llm evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems37, 68772– 68802 (2024)

    2024

  14. [14]

    arXiv preprint arXiv:2408.10343 (2024)

    Pipitone, N., Alami, G.H.: Legalbench-rag: A benchmark for retrieval-augmented generation in the legal domain. arXiv preprint arXiv:2408.10343 (2024)

    Pith/arXiv arXiv 2024

  15. [15]

    Qwen Team: Qwen3.5: Towards native multimodal agents (February 2026),https: //qwen.ai/blog?id=qwen3.5

    2026

  16. [16]

    arXiv preprint arXiv:2306.09525 (2023)

    Savelka, J., Ashley, K.D., Gray, M.A., Westermann, H., Xu, H.: Explaining legal concepts with augmented large language models (gpt-4). arXiv preprint arXiv:2306.09525 (2023)

    Pith/arXiv arXiv 2023

  17. [17]

    No permit required

    Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S., Dur- mus,E.,Hatfield-Dodds,Z.,Johnston,S.,Kravec,S.,etal.:Towardsunderstanding sycophancy in language models. In: International Conference on Learning Repre- sentations. vol. 2024, pp. 110–144 (2024) Evaluating RAG for French immigration law 9 Supplementary material A Benchmark buildi...

    arXiv 2024

  18. [18]

    Bilateral agreements

  19. [19]

    NOT eligible for CESEDA permit types

    Service-Public Critical rules: - EU/EEA/Swiss citizens: Check free movement rights first - Algerians: Apply Franco-Algerian Agreement (1968) for all substantive rules; CESEDA only for procedural gaps. NOT eligible for CESEDA permit types. - Metier en tension: Verify shortage occupation status -- significantly affects eligibility Tasks:

    1968

  20. [20]

    Check EU/EEA/Swiss status -> bilateral agreement France-{nationality } -> CESEDA eligibility articles

  21. [21]

    Verify metier en tension status

  22. [22]

    Find salary thresholds (Code du Travail / CESEDA carte Talent)

  23. [23]

    Determine qualifying permit type(s) and required documents (Service-Public / CESEDA)

  24. [24]

    general_recommendation

    Recommend the most appropriate permit with legal justification and document list Reason internally. Output only the final result as valid JSON in French, matching the provided output format. { "general_recommendation": "<type of permit recommended>", "justification": "<reasoning for why this permit type fits the applicant’s profile; no legal citations her...