An Empirical Investigation of Gender Stereotype Representation in Large Language Models: The Italian Case

Antonio Vetr\`o; Gioele Giachino; Juan Carlos De Martin; Marco Rondina; Riccardo Coppola

REVIEW 4 major objections 5 minor 23 references

An Empirical Investigation of Gender Stereotype Representation in Large Language Models: The Italian Case

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Given ungendered Italian prompts, ChatGPT and Gemini both attach leadership roles to male pronouns and subordinate roles to female pronouns at near-deterministic rates.

desk verdict A modest, reproducible Italian-language bias audit whose female-pronoun result is robust; the authors overstate the male-leadership finding and mislabel their prompts as ungendered. read the letter →

arxiv 2507.19156 v1 pith:K4K465DY submitted 2025-07-25 cs.CL cs.AIcs.CYcs.HC

Gioele Giachino , Marco Rondina , Antonio Vetrò , Riccardo Coppola , Juan Carlos De Martin This is my paper · ORCID

classification cs.CLcs.AIcs.CYcs.HC

keywords genderbiaslargelanguagemodelsItalianstereotypesprofessionalrolesconditionalprobabilityblack-boxauditingpromptdesign

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that two widely used commercial chatbots, OpenAI ChatGPT (gpt-4o-mini) and Google Gemini (gemini-1.5-flash), reproduce traditional gender stereotypes when answering ungendered prompts in Italian about professional hierarchies. Using zero-shot, no-context prompts that pair a leadership title ('manager', 'preside', 'chef') with a subordinate title ('assistente', 'insegnante', 'sous chef') and a gendered pronoun 'lui' or 'lei', the authors measured how often each pronoun is attached to each role across 3,600 API responses. They report that both models associate leadership roles with masculine pronouns and subordinate roles with feminine pronouns at near-deterministic rates: for instance, Gemini attached 100% (ChatGPT 97%) of 'she' answers to the assistant rather than the manager. The finding matters because these models are entering hiring, education, and public administration, where such associations could reinforce inequality, and because Italian's rich grammatical gender makes the test a sharp, non-English probe.

What carries the argument

The load-bearing mechanism is a controlled prompt corpus combined with two conditional probability metrics, $P(Y\mid B)$ (probability that a job title $Y$ is output given pronoun $B$) and $P(B\mid Y)$ (probability that a pronoun $B$ is used given output title $Y$). The prompts are zero-shot, no-context workplace scenarios: five templates, each instantiated with a hierarchical job pair and a gendered pronoun 'lui' or 'lei', in both title orders, producing 60 distinct prompts and 3,600 collected responses. The three job pairs — manager/assistant, preside/insegnante, chef/sous chef — were chosen as 'as neutral as possible' in Italian while preserving a clear hierarchy, so that an unbiased model would have no lexical reason to attach one pronoun to one title. The metrics convert raw co-occurrence counts into the probabilities the paper interprets as measurable gender bias.

What would settle it

Rerun the exact experiment with the leadership titles spelled with explicit feminine agreement in the prompt (e.g., 'la preside', 'la manager', 'la chef') while keeping the subordinate titles unchanged; if the models then attach 'lei' to the leadership roles at rates comparable to the original 'lui' rates, the leader-male association is substantially a product of Italian grammatical gender defaults rather than social stereotyping.

Watch

Extended reading notes

Core claim

The paper's central claim is that, given prompts with no gender marking on the job titles, both ChatGPT and Gemini behave as if professional seniority and gender were coupled: the higher-ranking role is treated as male and the lower-ranking one as female. The evidence is a set of conditional probabilities computed over $3{,}600$ responses. Across all three job pairs, the probability of the leadership title given a female pronoun is very low — for Gemini, $P(\text{manager}\mid\text{she})=0$, $P(\text{preside}\mid\text{she})=0$, $P(\text{chef}\mid\text{she})=0.07$ — while the complementary probabilities for subordinate titles with 'she' and leadership titles with 'he' are correspondingly high; ChatGPT shows the same direction with values such as $P(\text{manager}\mid\text{she})=0.03$ and $P(\text{assistant}\mid\text{she})=0.97$. The paper reads these numbers as models 'reflecting traditional gender norms' and notes the pattern is stable across both chatbots, with ChatGPT slightly more unbalanced than Gemini.

Load-bearing premise

The three job titles are genuinely gender-neutral in Italian as the prompts use them, so a bias-free model would have no statistical reason to attach 'lui' to the leader and 'lei' to the subordinate.

Editorial extensions

If this is right

In Italian-language uses of these chatbots (hiring drafts, career advice, reference letters), prompts that do not specify gender will tend to yield text placing men in leadership and women in support roles.
The bias is not an artifact of one vendor: two independently built proprietary systems show the same directional pattern, pointing to a shared root in training data or task formulation rather than a single company's fine-tuning.
Because the pattern appears under zero-shot, no-context prompting, users would need explicit counter-stereotypical context or mitigation to avoid it; adding neutral context alone may not suffice.
The conditional-probability protocol is a reproducible black-box audit that can be re-run on other models, languages, and job pairs without internal model access.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors do not separate grammatical from social causes: titles such as 'preside' and 'chef' may carry masculine lexical defaults in Italian training data, so part of the observed association could be morphological; rerunning the protocol with feminized agreement (e.g., 'la preside', 'la chef') would test this directly.
Extending the same battery to a language with little grammatical gender (e.g., English) or to Italian with gender-neutral endings (schwa) would reveal whether the near-deterministic pattern is driven by Italian morphology or by occupational stereotypes encoded in the training corpus.
Because the paper reports that the position of the job titles inside the prompt changes responses, the 'leadership equals male' effect may include a syntactic priming component; analysing the data separately by permutation, which the paper's repository makes possible, could estimate how much of the bias is order-driven.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The short version: this is a small, honest measurement paper on gender stereotypes in Italian LLM outputs. It is not a new phenomenon, and the paper cites the prior Italian work (Ruzzetti et al., 2023) that already established the general direction. Its value is concrete: reproducible, model-specific probabilities for gpt-4o-mini and gemini-1.5-flash in a WinoBias-style coreference setup with three hierarchical job pairs. The female-pronoun asymmetry is robust across all pairs: P(subordinate | lei) is between 0.89 and 1.00. That part of the conclusion holds.

What is actually new: the specific measurements for these two model versions, the Italian job-pair set, and a public repo with prompts, responses, and code. The conditional probability calculations are straightforward and the tables support the main empirical claim. This is a clean black-box audit with no fitted parameters and no reliance on the authors' own prior results.

Where I part ways with the paper and with the stress-test note. The stress-test says the 'constantly associating leadership roles with males' claim is contradicted by the JP2 results. I think that is too strong. Read as P(male | leader), the data do support the association: P(He|Manager)=1.00, P(He|Principal)=1.00, P(He|Chef)=0.89 for Gemini, and 0.97, 0.81, 0.84 for ChatGPT. What the JP2 data actually show is pair dependence: given a male pronoun, the model does not reliably pick the leader. The robust effect is female pronoun -> subordinate. The abstract and Section 4 overstate the male-leadership result, but it is not contradicted.

Real soft spots: (1) the prompts are called 'ungendered' even though each one contains an explicit lui/lei; only the job titles are intended to be neutral. That wording should be fixed. (2) No confidence intervals, significance tests, or decoding parameters are reported, so the probabilities look more exact than they are. (3) The ~2% of ambiguous responses are excluded with no sensitivity analysis; the paper asserts this does not affect validity but does not demonstrate it. (4) Only three job pairs limits generalizability, and the authors acknowledge this.

Who this is for: people working on non-English bias auditing, EU AI Act risk assessment, or Italian NLP. It is not a breakthrough, but it is a usable data point and the repo makes it easy to build on. I would send it to referees with an expectation of light-to-moderate revision on framing, statistical reporting, and the 'ungendered' terminology. I would engage with it as a referee.

Referee Report

4 major / 5 minor

Summary. The paper reports a black-box audit of two proprietary LLMs (OpenAI ChatGPT gpt-4o-mini and Google Gemini gemini-1.5-flash) for gender-stereotype associations in Italian. Using prompts built from three hierarchical job-title pairs (Manager-Assistant, Principal-Professor, Chef-Sous Chef), five base scenarios, two orders, and two pronouns (lui/lei), the authors collected 3,600 API responses and computed conditional probabilities P(role|pronoun) and P(pronoun|role). They report strong asymmetric associations, e.g. Gemini associating 'she' with Assistant rather than Manager 100% of the time, and conclude that both models reflect traditional gender norms. The paper also provides a public repository with code, prompts, and detailed results.

Significance. If the conclusions were stated accurately, this would be a useful, reproducible contribution to the small but growing literature on LLM bias in gendered non-English languages. The experimental design is transparent: model versions are named, the prompt inventory is fully listed, counts are reported, and the anonymous GitHub link makes replication straightforward. The paper also honestly acknowledges several limitations in Section 6. However, the central quantitative finding as stated is broader than the data support: the male-leadership association is not constant across job pairs, and the prompts are not ungendered because each embeds an explicit pronoun. With a corrected interpretation and uncertainty quantification, the study can still stand as a modest empirical data point, but its current headline claims overreach.

major comments (4)

[Section 3.2, Prompt Design; Section 4, Summary of RQ1; Abstract] The prompts are repeatedly described as 'ungendered', but every prompt contains an explicit gendered pronoun (lui/lei), e.g. Table 2, P1-A: 'perché lui era in ritardo'. The measured quantities are therefore anaphora-resolution probabilities P(role|pronoun): given that the model is told the late person is 'he' or 'she', which role does it pick? This is a substantially different construct from open-ended generation from a gender-neutral prompt, and it affects the interpretation of RQ1 and the abstract's claim that the study examines 'responses to ungendered prompts'. The terminology should be corrected throughout, and the discussion should be framed as pronoun-to-role association rather than unconstrained generation.
[Section 4, Summary of the Answer to RQ1; Abstract; Section 5] The claim that 'both Gemini and ChatGPT reflected traditional gender norms by constantly associating leadership roles with males and subordinate ones with women' is contradicted by the paper's own tables. For JP2 (Principal-Professor), the male pronoun selects the subordinate role (Professor) more often than the leadership role: Gemini P(Professor|He)=0.64 versus P(Principal|He)=0.36 (Table 4), and ChatGPT P(Professor|He)=0.68 versus P(Principal|He)=0.32 (Table 7). For JP3 (Chef-Sous Chef), the male-pronoun split is near balanced for both models (Gemini 0.54/0.46, ChatGPT 0.62/0.38). The only pattern that is consistent across all three pairs and both models is the female-pronoun-to-subordinate-role association (P>=0.89 everywhere). The abstract and Section 4 should be revised to state this pair-dependent pattern rather than claiming a constant male-leadership association.
[Section 4, Tables 3-8; Section 3.2, Bias Quantification Metrics] No confidence intervals or significance tests accompany any of the reported conditional probabilities, even though sample sizes are on the order of 300 per condition and exact counts are already in the tables. For example, Table 3 reports P(Manager|She)=0.00 from 296 observations, but without an interval the reader cannot assess the precision of this estimate or how it compares with, say, P(Manager|He)=0.69. The 2% anomaly exclusion is also reported only globally; the tables show that denominators vary (e.g. Table 4 has 293 in the 'she' column, Table 5 has 287), so the per-condition exclusion counts should be disclosed. Adding binomial confidence intervals or a simple test of association would materially strengthen the paper's claims of systematic bias.
[Section 3.2, Job Pair Selection] The load-bearing premise that the three job-title pairs are 'as neutral as possible' in Italian is asserted without linguistic evidence or a control. If any title carries a masculine lexical default (for instance, 'chef' and 'sous chef' are grammatically masculine nouns in Italian), then part of the Chef-He association in JP3 could reflect grammatical gender rather than social stereotyping. This is especially relevant because the observed male-pronoun associations are the weakest and most inconsistent findings. The authors should either provide evidence for lexical neutrality of the Italian titles or explicitly acknowledge that the design cannot separate grammatical gender from stereotypical association, and temper the conclusions accordingly.

minor comments (5)

[Section 4.1, Google Gemini, JP2 bullet] The text says 'in a way similar to JP2' but the preceding discussion concerns JP1; this should read 'in a way similar to JP1'.
[Tables 3-8] The column headers are dense and partly redundant (e.g. 'Y = he(lui)' followed by three sub-columns each labeled 'he(lui)'). The tables would be much easier to read if the top header distinguished 'counts' from 'P(Y|B)' and 'P(B|Y)' explicitly, and if the total row were separated from the data rows.
[Abstract] The abstract says 'a range of 3600 responses' but does not report how many responses were excluded as anomalies or how many entered the probability calculations; since the tables show varying totals, the final usable count should be stated.
[Section 3.2, Experimental Setup] The sentence 'To adhere to rate constraints and preserve reproducibility, brief delays (sleep) were added in between API calls' is good practice, but the paper should also state the API access dates and the exact model snapshot identifiers returned by the APIs, since proprietary models are updated over time.
[Section 2, Related Work] The citation to Kotek et al. [5] says LLMs are '3-6 times more likely to assign stereotypical professional roles when forced to answer in a gendered manner', but the precise conditions under which this ratio was measured are not summarized; a one-sentence context would help the reader judge comparability with this study.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports direct black-box measurements of LLM outputs with no fitted parameters, no self-citation chain, and no prediction derived from its own inputs.

full rationale

This is an empirical audit, not a derivation. The chain is: select job pairs and prompts, collect API responses via repeated submissions, count role-pronoun co-occurrences, and report conditional probabilities. Every reported table value is a direct aggregate of collected responses; no parameter is fitted, no quantity is predicted from a subset and then validated on a related subset, and no uniqueness theorem or same-author prior result is invoked. The only external design influence is WinoBias, cited for the prompt schema, which is independent and not load-bearing for the quantitative claim. The paper's own limitations section acknowledges a construct-validity caveat ('Even if the prompts were intended to be ungendered, latent distributional patterns or training-specific preferences may nevertheless have an impact on how LLMs read them'), but this is a validity risk, not a circular reduction. The Section 4 summary's word 'constantly' overstates some pair-specific results (e.g., Gemini JP2 has P(Principal|He)=0.36 vs P(Professor|He)=0.64), and the prompts are called 'ungendered' while including an explicit lui/lei pronoun; these are accuracy and terminology concerns, not circularity. No step reduces, by construction, to the paper's own inputs, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities. Four domain assumptions carry the measurement: lexical neutrality of the job titles, the isolating power of the prompt design, the independence of the anomaly exclusion, and the unstated stochastic sampling of the APIs.

assumptions (4)

domain assumption The three job title pairs (manager/assistant, principal/professor, chef/sous chef) are gender-neutral in Italian and equally compatible with masculine and feminine pronouns.
Section 3.2 'Job Pair Selection' states the titles are chosen 'as neutral as possible'; if any title carries a lexical gender default, the measured pronoun-role association conflates stereotype with grammar.
domain assumption The WinoBias-style prompts with an explicit gendered pronoun (lui/lei) isolate stereotype-driven role selection rather than scenario-driven expectations about who would be late, present, receive an award, etc.
Section 3.2 'Prompt Design' varies pronoun gender and role order while keeping the scenario fixed; the design assumes the only systematic driver is pronoun gender, not scenario-role plausibility.
domain assumption The excluded 'anomaly' responses are independent of pronoun gender and job pair, so their removal does not bias the conditional probabilities.
Section 3.2 'Anomalies handling' discards vague replies such as a bare 'Lei' (less than 2% of responses) with informal criteria and no sensitivity analysis.
domain assumption The API queries were run with stochastic sampling (temperature > 0) so that 30 iterations per prompt estimate the model's output distribution.
Section 3.2 'Experimental Setup' reports 30 submissions per prompt but does not report temperature, top-p, or other decoding parameters; the non-multiple-of-30 counts in the tables suggest stochasticity, but this is unstated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Empirical Investigation of Gender Stereotype Representation in Large Language Models: The Italian Case." pith.science (2026). https://pith.science/paper/K4K465DY

@misc{pith2026250719156,
  author       = {Pith},
  title        = {Pith review of: An Empirical Investigation of Gender Stereotype Representation in Large Language Models: The Italian Case},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K4K465DY}},
  note         = {Machine review of arXiv:2507.19156}
}

read the original abstract

The increasing use of Large Language Models (LLMs) in a large variety of domains has sparked worries about how easily they can perpetuate stereotypes and contribute to the generation of biased content. With a focus on gender and professional bias, this work examines in which manner LLMs shape responses to ungendered prompts, contributing to biased outputs. This analysis uses a structured experimental method, giving different prompts involving three different professional job combinations, which are also characterized by a hierarchical relationship. This study uses Italian, a language with extensive grammatical gender differences, to highlight potential limitations in current LLMs' ability to generate objective text in non-English languages. Two popular LLM-based chatbots are examined, namely OpenAI ChatGPT (gpt-4o-mini) and Google Gemini (gemini-1.5-flash). Through APIs, we collected a range of 3600 responses. The results highlight how content generated by LLMs can perpetuate stereotypes. For example, Gemini associated 100% (ChatGPT 97%) of 'she' pronouns to the 'assistant' rather than the 'manager'. The presence of bias in AI-generated text can have significant implications in many fields, such as in the workplaces or in job selections, raising ethical concerns about its use. Understanding these risks is pivotal to developing mitigation strategies and assuring that AI-based systems do not increase social inequalities, but rather contribute to more equitable outcomes. Future research directions include expanding the study to additional chatbots or languages, refining prompt engineering methods or further exploiting a larger experimental base.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 11 canonical work pages

[1]

In:AdvancesinNeuralInformationProcessingSystems.vol.29.CurranAssociates, Inc

Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V., Kalai, A.T.: Man is to ComputerProgrammerasWomanistoHomemaker?DebiasingWordEmbeddings. In:AdvancesinNeuralInformationProcessingSystems.vol.29.CurranAssociates, Inc. (2016), https://proceedings.neurips.cc/paper_files/paper/2016/hash/ a486cd07e4ac3d270571622f4f316ec5-Abstract.html

work page 2016
[2]

https://doi

Chaudhary, Y., Penn, J.: Large Language Models as Instruments of Power: New Regimes of Autonomous Manipulation and Control (May 2024). https://doi. org/10.48550/arXiv.2405.03813

arXiv doi:10.48550/arxiv.2405.03813 2024
[3]

europa.eu/eli/reg/2024/1689/oj/eng

European Parliament, European Council: Regulation (EU) 2024/1689 of the Eu- ropean Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and...

work page 2024
[4]

First Monday (Nov 2023).https://doi.org/10.5210/fm.v28i11

Ferrara, E.: Should ChatGPT be biased? Challenges and risks of bias in large lan- guage models. First Monday (Nov 2023).https://doi.org/10.5210/fm.v28i11. 13346

work page doi:10.5210/fm.v28i11 2023
[5]

In: Proceedings of The ACM Collective Intelligence Conference

Kotek, H., Dockum, R., Sun, D.: Gender bias and stereotypes in Large Language Models. In: Proceedings of The ACM Collective Intelligence Conference. pp. 12–24. CI ’23, Association for Computing Machinery, New York, NY, USA (Nov 2023). https://doi.org/10.1145/3582269.3615599

arXiv 2023
[6]

In: Proceedings of the 2024 ACM Conference on Fair- ness, Accountability, and Transparency

Liesenfeld, A., Dingemanse, M.: Rethinking open source generative ai: open- washing and the eu ai act. In: Proceedings of the 2024 ACM Conference on Fair- ness, Accountability, and Transparency. p. 1774–1787. FAccT ’24, Association for Computing Machinery, New York, NY, USA (2024).https://doi.org/10.1145/ 3630106.3659005, https://doi.org/10.1145/3630106.3659005

arXiv 2024
[7]

A "Perspectival" Mirror of the Elephant: Investigating Language Bias on Google, ChatGPT, YouTube, and Wikipedia

Luo, Q., Puett, M.J., Smith, M.D.: A "perspectival" mirror of the elephant: Investigating language bias on google, chatgpt, youtube, and wikipedia (2024), https://arxiv.org/abs/2303.16281

work page Pith review arXiv 2024
[8]

https://doi.org/10.48550/arXiv.2502.02289

Magnini, B., Zanoli, R., Resta, M., Cimmino, M., Albano, P., Madeddu, M., Patti, V.: Evalita-LLM: Benchmarking Large Language Models on Italian (Feb 2025). https://doi.org/10.48550/arXiv.2502.02289

arXiv doi:10.48550/arxiv.2502.02289 2025

Show all 23 references

[9]

Maina, H., Alemany, L.A., Ivetta, G., Rajngewerc, M., Busaniche, B., Benotti, L.: Exploring Stereotypes and Biases in Language Technologies in Latin America. Commun. ACM 67(8), 54–56 (Aug 2024).https://doi.org/10.1145/3653322

2024 doi
[10]

https://doi.org/10.48550/arXiv.2406.17535 An Empirical Investigation of Gender Stereotype Representation in LLMs 15

Mercorio, F., Mezzanzanica, M., Potertì, D., Serino, A., Seveso, A.: Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmark (Jun 2024). https://doi.org/10.48550/arXiv.2406.17535 An Empirical Investigation of Gender Stereotype Representation in LLMs 15

2024 doi arXiv
[11]

In: Chiruzzo, L., Ritter, A., Wang, L

Mitchell, M., Attanasio, G., Baldini, I., Clinciu, M., Clive, J., Delobelle, P., Dey, M., Hamilton, S., Dill, T., Doughman, J., Dutt, R., Ghosh, A., Forde, J.Z., Holtermann, C., Kaffee, L.A., Laud, T., Lauscher, A., Lopez-Davila, R.L., Ma- soud, M., Nangia, N., Ovalle, A., Pis...

2025
[12]

In: ICML 2024 Next Generation of AI Safety Workshop (Jul 2024),https://openreview

Morehouse,K.,Pan,W.,Contreras,J.M.,Banaji,M.R.:BiasTransmissioninLarge Language Models: Evidence from Gender-Occupation Bias in GPT-4. In: ICML 2024 Next Generation of AI Safety Workshop (Jul 2024),https://openreview. net/forum?id=Fg6qZ28Jym

2024
[13]

Naveed, H., Khan, A.U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., Mian, A.: A comprehensive overview of large language models (2024), https://arxiv.org/abs/2307.06435

2024 arXiv
[14]

Navigli, R., Conia, S., Ross, B.: Biases in large language models: Origins, inventory, and discussion. J. Data and Information Quality15(2) (Jun 2023).https://doi. org/10.1145/3597307, https://doi.org/10.1145/3597307

2023 doi
[15]

In: Muresan, S., Nakov, P., Villavicencio, A

Névéol, A., Dupont, Y., Bezançon, J., Fort, K.: French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting o...

2022 doi
[16]

In: Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., Schock- aert, S

Puccetti, G., Cassese, M., Esuli, A.: The Invalsi Benchmarks: Measuring the Lin- guistic and Mathematical understanding of Large Language Models in Italian. In: Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., Schock- aert, S. (eds.) Proceedings of the 31...

2025
[17]

Rini van Solingen, Basili, V., Caldiera, G., Rombach, H.D.: Goal Question Metric (GQM) Approach. In: J.J. Marciniak (ed.) Encyclopedia of Software Engineering. John Wiley & Sons, USA (2002).https://doi.org/10.1002/0471028959.sof142

2002 doi
[18]

In: CLiC-it 2023: 9th Italian Conference on Computational Linguistics (2023)

Ruzzetti, E.S., Onorati, D., Ranaldi, L., Venditti, D., Zanzotto, F.M.: Investigating Gender Bias in Large Language Models for the Italian Language. In: CLiC-it 2023: 9th Italian Conference on Computational Linguistics (2023)

2023
[19]

Sahoo, P., Singh, A.K., Saha, S., Jain, V., Mondal, S., Chadha, A.: A System- atic Survey of Prompt Engineering in Large Language Models: Techniques and Applications (Mar 2025).https://doi.org/10.48550/arXiv.2402.07927

2025 doi arXiv
[20]

Giachino, M

Thellmann, K., Stadler, B., Fromm, M., Buschhoff, J.S., Jude, A., Barth, F., Leveling, J., Flores-Herr, N., Köhler, J., Jäkel, R., Ali, M.: Towards Multilin- 16 G. Giachino, M. Rondina et al. gual LLM Evaluation for European Languages (Oct 2024).https://doi.org/10. 48550/arXiv...

2024 doi arXiv
[21]

Kelly is a Warm Person, Joseph is a Role Model

Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.W., Peng, N.: "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters (Dec 2023).https://doi.org/10.48550/arXiv.2310.09219

2023 doi arXiv
[22]

Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods (Apr 2018).https: //doi.org/10.48550/arXiv.1804.06876

2018 doi arXiv
[23]

Zhou, K.Z., Sanfilippo, M.R.: Public Perceptions of Gender Bias in Large Language Models: Cases of ChatGPT and Ernie (Sep 2023).https://doi.org/10.48550/ arXiv.2309.09120

2023 doi arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

In:AdvancesinNeuralInformationProcessingSystems.vol.29.CurranAssociates, Inc

Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V., Kalai, A.T.: Man is to ComputerProgrammerasWomanistoHomemaker?DebiasingWordEmbeddings. In:AdvancesinNeuralInformationProcessingSystems.vol.29.CurranAssociates, Inc. (2016), https://proceedings.neurips.cc/paper_files/paper/2016/hash/ a486cd07e4ac3d270571622f4f316ec5-Abstract.html

work page 2016

[2] [2]

https://doi

Chaudhary, Y., Penn, J.: Large Language Models as Instruments of Power: New Regimes of Autonomous Manipulation and Control (May 2024). https://doi. org/10.48550/arXiv.2405.03813

arXiv doi:10.48550/arxiv.2405.03813 2024

[3] [3]

europa.eu/eli/reg/2024/1689/oj/eng

European Parliament, European Council: Regulation (EU) 2024/1689 of the Eu- ropean Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and...

work page 2024

[4] [4]

First Monday (Nov 2023).https://doi.org/10.5210/fm.v28i11

Ferrara, E.: Should ChatGPT be biased? Challenges and risks of bias in large lan- guage models. First Monday (Nov 2023).https://doi.org/10.5210/fm.v28i11. 13346

work page doi:10.5210/fm.v28i11 2023

[5] [5]

In: Proceedings of The ACM Collective Intelligence Conference

Kotek, H., Dockum, R., Sun, D.: Gender bias and stereotypes in Large Language Models. In: Proceedings of The ACM Collective Intelligence Conference. pp. 12–24. CI ’23, Association for Computing Machinery, New York, NY, USA (Nov 2023). https://doi.org/10.1145/3582269.3615599

arXiv 2023

[6] [6]

In: Proceedings of the 2024 ACM Conference on Fair- ness, Accountability, and Transparency

Liesenfeld, A., Dingemanse, M.: Rethinking open source generative ai: open- washing and the eu ai act. In: Proceedings of the 2024 ACM Conference on Fair- ness, Accountability, and Transparency. p. 1774–1787. FAccT ’24, Association for Computing Machinery, New York, NY, USA (2024).https://doi.org/10.1145/ 3630106.3659005, https://doi.org/10.1145/3630106.3659005

arXiv 2024

[7] [7]

A "Perspectival" Mirror of the Elephant: Investigating Language Bias on Google, ChatGPT, YouTube, and Wikipedia

Luo, Q., Puett, M.J., Smith, M.D.: A "perspectival" mirror of the elephant: Investigating language bias on google, chatgpt, youtube, and wikipedia (2024), https://arxiv.org/abs/2303.16281

work page Pith review arXiv 2024

[8] [8]

https://doi.org/10.48550/arXiv.2502.02289

Magnini, B., Zanoli, R., Resta, M., Cimmino, M., Albano, P., Madeddu, M., Patti, V.: Evalita-LLM: Benchmarking Large Language Models on Italian (Feb 2025). https://doi.org/10.48550/arXiv.2502.02289

arXiv doi:10.48550/arxiv.2502.02289 2025