REVIEW 4 major objections 5 minor 30 references
Toward Inclusive Educational AI: Auditing Frontier LLMs through a Multiplexity Lens
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper argues that cultural bias in LLM educational responses can be measured with a reference-distribution score and reduced from 3.25% to 98% normalized entropy by a multi-agent multiplexity pipeline, without retraining.
desk verdict The 98% PDS entropy is built into the MAS protocol, not evidence of inclusivity; the paper is a useful pilot but the central claim is unsupported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Perspectives Distribution Score (PDS): a vector where each entry is one culture's share of all cultural references extracted from a response, with PDS Entropy defined as the normalized Shannon entropy of that vector. That metric turns free-text answers into a number between 0% and 100%, where 100% means perfectly even coverage. Around it, the framework is built from three devices: a zero-shot extractor and sentiment classifier; a system-prompt version that embeds multiplex ontology and multiplex epistemology instructions; and a multi-agent version where eight culturally positioned agents answer, then a multiplex agent synthesizes their outputs according to multiplexity rules. The PDS is what lets the paper compare baseline, prompted, and agent-synthesized outputs on the same scale.
What would settle it
Have independent human annotators label cultural references in the same 47-question responses and recompute both the Perspectives Distribution Score and its entropy; if baseline entropy under human labels is well above 3.25% or the multi-agent output falls far below 98%, the reported effect is an artifact of the extractor rather than a property of the models. A simpler variant is to swap the extractor for a different model or a dictionary-based matcher and check whether the 3.25%-to-98% gap persists.
Extended reading notes
Core claim
The central discovery is that cultural inclusivity in LLM-generated answers behaves like a prompt-controllable property rather than a fixed model trait. The paper defines cultural bias operationally as skew in the Perspectives Distribution Score, and reports average normalized PDS entropy of 3.25% for default models, 19% when multiplexity rules are embedded in the system prompt, and 98% when a coordinator-facilitated workforce of culturally persona'd agents produces answers that a multiplex agent synthesizes. The paper reads these numbers as showing that multiplex models become markedly more inclusive, with the MAS version approximating the ideal 12.5% share for each of eight perspectives and achieving 0% negative sentiment in the accompanying bias-framing analysis.
Load-bearing premise
The whole measurement rests on treating how many cultural references a computer model finds in a response as the definition of cultural inclusivity: an answer with no cultural references counts as biased, and an answer mentioning all eight cultures counts as inclusive.
Editorial extensions
If this is right
- Default frontier models, measured on 47 educational questions, concentrate their cultural references in Western perspectives, with average normalized PDS entropy of only 3.25%.
- Adding multiplexity principles to the system prompt, without changing the underlying model, lifts average entropy to 19% and shifts sentiment away from neutral toward positive.
- A multi-agent implementation in which each culture is represented by a persona'd agent and a multiplex agent synthesizes their answers achieves near-uniform coverage, 98% entropy, and 0% negative sentiment.
- These gains are reached at inference time, so pluralistic educational content can be produced without fine-tuning or retraining, at the cost of orchestrating multiple agents.
Reading between the lines
- An editor-level extension of this result is that the 'Others' category may be absorbing many underrepresented perspectives; a finer cultural taxonomy could lower the 98% figure by revealing uneven coverage hiding inside that bucket.
- Because the same model family is used as generator, extractor, and sentiment judge, part of the reported improvement could reflect the evaluator's sensitivity to prompt-induced references; testing with an independent classifier would separate prompt compliance from genuine cultural grounding.
- The framework suggests a practical deployment-time alternative to fine-tuning, but it also implies that inclusivity is being scored by surface reference coverage; combining PDS with task-level measures would test whether the diversity is substantive.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a multiplexity-inspired framework for auditing and mitigating cultural bias in large language models (LLMs) used in education. The authors define eight cultural perspectives, assemble 47 educational questions across eight topic categories, and evaluate four LLMs (GPT-4o, Claude 3.5 Sonnet, Llama 3.1, and Mistral 7B) under three conditions: a baseline, a contextually implemented multiplexity prompt, and a multi-agent system (MAS) in which agents with distinct cultural personas answer and a Multiplex Agent synthesizes the responses. Cultural references are extracted by a GPT-4o zero-shot classifier and summarized by the Perspectives Distribution Score (PDS) and PDS entropy. The paper reports a baseline PDS entropy of 3.25%, rising to 19% under contextual multiplexity and 98% under the MAS implementation, and claims that this demonstrates markedly improved cultural inclusivity. Sentiment analysis is used to support a further claim that the MAS strategy shifts sentiment toward positive across cultures, reaching 0% negative sentiment.
Significance. If the central result were valid, it would be practically significant: it would suggest that a prompt-level, multi-agent intervention can make LLM outputs reference a near-uniform spread of eight cultural perspectives without fine-tuning, which is directly relevant to global educational AI. The paper also provides a concrete evaluation scaffold—an explicit cultural taxonomy, a normalized distribution metric, and a reproducible (if not yet released) extraction-and-scoring pipeline—and it is transparent about the OpenAI-only limitation of its MAS implementation. However, the empirical foundation is currently too weak to support the headline claim. The measurement instrument is unvalidated, and the MAS condition largely encodes the desired output distribution in its instructions, so the 3.25%-to-98% jump does not establish a change in the models' underlying cultural inclusivity.
major comments (4)
- [§III-C3, Box 2, Box 3, and §IV-B.2] The headline result—PDS entropy rising from 3.25% to 98%—is largely a consequence of the MAS protocol rather than a measured change in the model's cultural inclusivity. In Strategy 3, each agent is explicitly assigned a cultural persona and instructed to write from that perspective (Box 2), and the WorkForce template directs the system to ask every agent to write on the topic and then have the Multiplex Agent synthesize all perspectives (Box 3). The resulting output is therefore expected to contain at least one passage associated with each of the eight categories, so a PDS vector near 12.5% per culture and entropy near 100% is nearly forced. The comparison in Table III is not like-for-like: baseline and contextual prompts do not instruct the model to mention each culture, while the MAS prompt does. The abstract's causal claim that 'cultural inclusivity markedly improves' as mitigation evolves is not supported by this contrast.
- [§III-C4 and §III-C5] The PDS is built entirely from mention counts produced by a GPT-4o zero-shot Perspectives Extractor, and the paper reports no human agreement data, no confidence intervals, and no error analysis. Because the same model family is being audited and used as the measurement instrument, the baseline 'low diversity' scores may reflect the extractor's threshold for what counts as a cultural reference—especially in mathematics, lab, and optimization questions, which naturally contain few explicit cultural markers—rather than a property of the audited LLMs. The central baseline-bias claim (RQ1) therefore requires independent validation of the extractor before it can be accepted.
- [§III-C6 and §IV-C] The sentiment-analysis component is used to support RQ3, but it rests on two unvalidated assumptions: that GPT-4o zero-shot sentiment labels are accurate, and that a neutral label 'likely means' the LLM does not treat that culture equally. The latter interpretive rule is asserted without evidence and conflates absence of expressed sentiment with negative treatment. As a result, the reported shift from neutral to positive sentiment under the multiplex strategies does not establish improved cultural sensitivity.
- [§III-C3 and Table III] The MAS evaluation is performed only with GPT-4o, because Camel AI is reported to support only OpenAI models; the other three models have no MAS row in Table III. This means the paper's cross-model conclusion that MAS-implemented multiplexity improves inclusivity over contextual prompting is based on a single model, and no statistical comparison across the 47 questions is provided. The abstract's claim that 'as mitigation strategies evolve ... cultural inclusivity markedly improves' is therefore broader than the data support.
minor comments (5)
- [§III-C5] The section heading reads 'Perspectives Distribution Extractor (PDS)' although the section defines the Perspectives Distribution Score; the title should be corrected to avoid confusion between the extractor and the score.
- [§III-C4 and §III-C6] The full prompt templates for the Perspectives Extractor and the Sentiment Analyzer are not included; the paper only describes them in prose. Including the exact prompts would substantially improve reproducibility.
- [Table III] The dashes for the MAS-implemented multiplexity rows for Claude, Llama, and Mistral should be explained in the table caption, not only in the body text, since the caption currently makes the table appear incomplete.
- [Fig. 4] The pie charts in Figures 4a and 4b are very small, and the percentage labels are difficult to read; a supplementary table of the PDS vectors would make the distributions easier to inspect.
- [Box 1 and Table II] The exact values of MAX_WORDS and MAX_WORD_PER_AGENT are not stated, and the 47 questions are not listed; providing the question set and word limits in an appendix would allow readers to assess the breadth and neutrality of the prompts.
Circularity Check
The 3.25% to 98% PDS entropy gain is entailed by the MAS protocol, which requires each culturally assigned agent to write and the Multiplex Agent to synthesize all perspectives; PDS then counts those forced references.
-
self definitional
[§III-C3 (Strategy 3), Box 3; measured by §III-C5 PDS; reported in Table III and §IV-B2]
"You are given different agents representing various cultural perspectives. You are tasked to write exactly {MAX_WORDS} words on the topic {"TOPIC"}. First, ask each agent to write {MAX_WORD_PER_AGENT} words of content on the topic except for the Multiplex agent. Then, take the output of all the agents and pass it to the Multiplex Agent to write a synthesis of the different perspectives presented by all the other agents based on the rules of multiplexity that are already given to this agent."
Under Strategy 3, the audited output L_MA(Q) is constructed by prompting eight agents with distinct cultural personas (Box 2) to write content, then instructing the Multiplex Agent to synthesize "the different perspectives presented by all the other agents." The PDS metric (§III-C5) then counts references to the eight cultures in that synthesized text and normalizes them, so a near-uniform 12.5% per culture and 98% entropy are consequences of the task template, not of the underlying LLM's cultural inclusivity. Comparing this to baseline L(Q), where the Box 1 template asks only for "accurate answers" to math and lab questions (with no instruction to mention cultures), the 3.25% to 98% jump measures prompt compliance with a protocol that contains the measured diversity.
-
self definitional
[§III-C6 (Sentiment Analyzer - Bias Framing) applied to Strategy 3 output; results in §IV-C and Fig. 5b]
"However, a neutral sentiment does not necessarily indicate a positive stance toward a particular culture; it may also imply that the LLM does not consider that culture important or worthy of positive regard. Thus, if an LLM's response is neutral toward a culture, it likely means it is not treating that culture equally."
The coding rule redefines absence of cultural mention (neutral sentiment) as bias, so unconstrained baseline answers—especially for the 15 questions in Mathematical and Lab Related Topics, Table II—are scored as non-inclusive by construction. The MAS protocol then forces every agent to write culturally framed content, so the subsequent "shift towards positive sentiment" and "0% negative sentiment" (Fig. 5b) reflect the prompt's instruction to include and synthesize all perspectives rather than an independent improvement in how the models treat cultures.
full rationale
The paper's headline quantitative claim—that cultural inclusivity improves from 3.25% PDS entropy at baseline to 98% with MAS-implemented multiplexity—rests on measuring PDS on outputs produced by a pipeline whose prompt explicitly requires each of the eight cultural agents to write content and the Multiplex Agent to synthesize all those perspectives. Because PDS is defined as each culture's share of counted references (§III-C5, Pi = Ri / ΣRj), the near-uniform distribution is entailed by the prompt structure rather than discovered in the model's behavior. The baseline is not a like-for-like control: Box 1 asks for a bare "accurate answer," so math and lab responses naturally contain zero cultural references, and the paper's own sentiment-coding rule (§III-C6) treats neutral/no-mention responses as evidence of unequal treatment. Thus the quantitative comparison is largely a tautology: the intervention inserts the measured content and the metric counts what was inserted. The multiplexity framing itself is imported from the authors' own prior work (refs. [16]–[18], with Qadir an author of the present paper and of [17]–[18]), but that is not the load-bearing circular step; the load-bearing issue is the MAS protocol/PDS definition. Independent external validation (e.g., human evaluation of the GPT-4o Perspectives Extractor, or measuring PDS on outputs where cultural mentions are not prompted) would be needed to give the 98% figure non-circular content. Score 8: the central result is forced by definition and by the protocol construction.
Assumptions & free parameters
free parameters (4)
- MAX_WORDS prompt limit
- Eight-category cultural taxonomy =
8 perspectives including 'Others'
- Question set size and composition =
47 questions, 5 or 12 per category
- LLM decoding parameters
assumptions (5)
- domain assumption Multiplexity, as defined by Senturk et al. [16] and Qadir [17], is the correct normative standard for cultural inclusivity.
- domain assumption GPT-4o zero-shot classification of cultural references and sentiment is accurate.
- domain assumption Absence of cultural references in an educational answer is evidence of cultural bias.
- domain assumption A synthesized answer produced by one agent per culture is still a valid educational answer.
- domain assumption The 47 question set and eight cultural categories represent the world's cultural diversity.
invented entities (3)
-
Perspectives Distribution Score (PDS)
-
Multiplex Agent
-
Perspectives Extractor
Cite this review
Pith. "Pith review of Toward Inclusive Educational AI: Auditing Frontier LLMs through a Multiplexity Lens." pith.science (2026). https://pith.science/paper/XPVZU4A4
@misc{pith2026250103259,
author = {Pith},
title = {Pith review of: Toward Inclusive Educational AI: Auditing Frontier LLMs through a Multiplexity Lens},
year = {2026},
howpublished = {\url{https://pith.science/paper/XPVZU4A4}},
note = {Machine review of arXiv:2501.03259}
}
read the original abstract
As large language models (LLMs) like GPT-4 and Llama 3 become integral to educational contexts, concerns are mounting over the cultural biases, power imbalances, and ethical limitations embedded within these technologies. Though generative AI tools aim to enhance learning experiences, they often reflect values rooted in Western, Educated, Industrialized, Rich, and Democratic (WEIRD) cultural paradigms, potentially sidelining diverse global perspectives. This paper proposes a framework to assess and mitigate cultural bias within LLMs through the lens of applied multiplexity. Multiplexity, inspired by Senturk et al. and rooted in Islamic and other wisdom traditions, emphasizes the coexistence of diverse cultural viewpoints, supporting a multi-layered epistemology that integrates both empirical sciences and normative values. Our analysis reveals that LLMs frequently exhibit cultural polarization, with biases appearing in both overt responses and subtle contextual cues. To address inherent biases and incorporate multiplexity in LLMs, we propose two strategies: \textit{Contextually-Implemented Multiplex LLMs}, which embed multiplex principles directly into the system prompt, influencing LLM outputs at a foundational level and independent of individual prompts, and \textit{Multi-Agent System (MAS)-Implemented Multiplex LLMs}, where multiple LLM agents, each representing distinct cultural viewpoints, collaboratively generate a balanced, synthesized response. Our findings demonstrate that as mitigation strategies evolve from contextual prompting to MAS-implementation, cultural inclusivity markedly improves, evidenced by a significant rise in the Perspectives Distribution Score (PDS) and a PDS Entropy increase from 3.25\% at baseline to 98\% with the MAS-Implemented Multiplex LLMs. Sentiment analysis further shows a shift towards positive sentiment across cultures,...
Figures
Reference graph
Works this paper leans on
-
[1]
On the opportunities and risks of foundation models,
R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021
arXiv 2021
-
[2]
ChatGPT: Transforming the Future of AI,
OpenAI, “ChatGPT: Transforming the Future of AI,” 2024. [Online]. Available: https://openai.com/chatgpt
work page 2024
-
[3]
Claude 3.5: Introducing Sonnet,
A. AI, “Claude 3.5: Introducing Sonnet,” 2024. [Online]. Available: https://www.anthropic.com/news/claude-3-5-sonnet
work page 2024
-
[4]
Gemini: Revolutionizing AI with multimodal capabilities,
DeepMind, “Gemini: Revolutionizing AI with multimodal capabilities,”
-
[5]
M. AI, “The llama 3 herd of models,” 2024. [Online]. Available: https://ai.meta.com/llama/3
work page 2024
-
[6]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Łukasz Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems , vol. 30,
-
[7]
Data-driven artificial intelligence in education: A comprehensive review,
K. Ahmad, W. Iqbal, A. El-Hassan, J. Qadir, D. Benhaddou, M. Ayyash, and A. Al-Fuqaha, “Data-driven artificial intelligence in education: A comprehensive review,” IEEE Transactions on Learning Technologies , 2023
work page 2023
-
[8]
Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education,
J. Qadir, “Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education,” in 2023 IEEE Global Engineering Education Conference (EDUCON) . IEEE, 2023, pp. 1–9
2023
Show all 30 references
-
[9]
Meet khanmigo: Khan academy’s ai-powered teaching assistant & tutor
K. Academy, “Meet khanmigo: Khan academy’s ai-powered teaching assistant & tutor.” [Online]. Available: https://www.khanmigo.ai/
-
[10]
Teaching cs50 with AI,
R. Liu, C. Zenke, C. Liu, A. Holmes, P. Thornton, and D. J. Malan, “Teaching cs50 with AI,” Portland, OR, US: ACM , 2024
2024
-
[11]
Persistent anti-muslim bias in large language models,
A. Abid, M. Farooqi, and J. Zou, “Persistent anti-muslim bias in large language models,” arXiv preprint arXiv:2101.05783 , 2021. [Online]. Available: https://arxiv.org/abs/2101.05783
2021 arXiv
-
[12]
Ethical and social risks of harm from language models,
L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh et al., “Ethical and social risks of harm from language models,” arXiv preprint arXiv:2112.04359, 2021
2021 arXiv
-
[13]
Bias and fairness in chatbots: An overview,
J. Xue, Y .-C. Wang, C. Wei, X. Liu, J. Woo, and C.-C. J. Kuo, “Bias and fairness in chatbots: An overview,” arXiv preprint arXiv:2309.08836 , 2023
2023 arXiv
-
[14]
The ghost in the ma- chine has an american accent: value conflict in gpt-3,
R. L. Johnson, G. Pistilli, N. Men ´edez-Gonz´alez, L. D. D. Duran, E. Panai, J. Kalpokiene, and D. J. Bertulfo, “The ghost in the ma- chine has an american accent: value conflict in gpt-3,” arXiv preprint arXiv:2203.07785, 2022
2022 arXiv
-
[15]
Cultural bias and cultural alignment of large language models,
Y . Tao, O. Viberg, R. S. Baker, and R. F. Kizilcec, “Cultural bias and cultural alignment of large language models,” PNAS nexus, vol. 3, no. 9, p. pgae346, 2024. [Online]. Available: https://arxiv.org/abs/2311.14096
2024 arXiv
-
[16]
Comparative theories and methods: between uniplexity and multiplexity,
R. S ¸ent¨urk, A. Ac ¸ıkgenc ¸,¨O. K ¨uc ¸¨ukural, Q. N. Yamamoto, and N. K. Aksay, “Comparative theories and methods: between uniplexity and multiplexity,” 2020
2020
-
[17]
A holistic education for the 21st century engineer based on wisdom and multiplexity,
J. Qadir, “A holistic education for the 21st century engineer based on wisdom and multiplexity,” Transdisciplinarity and the Future of Engineering , 2022. [Online]. Available: https://doi.org/10.3233/ ATDE220694
2022
-
[18]
Educating for the AI era: Harnessing the wild genai horse through multiplex AI humanities and critical AI literacy,
J. Qadir and R. Senturk, “Educating for the AI era: Harnessing the wild genai horse through multiplex AI humanities and critical AI literacy,” 2024, available at SSRN: https://ssrn.com/abstract=5015109
2024
-
[19]
Training responsible engineers: Phronesis and the role of virtues in teaching engineering ethics,
G. Frigo, F. Marthaler, A. Albers, S. Ott, and R. Hillerbrand, “Training responsible engineers: Phronesis and the role of virtues in teaching engineering ethics,” Australasian Journal of Engineering Education , vol. 26, no. 1, pp. 25–37, 2021
2021
-
[20]
Analyzing student prompts and their effect on chatgpt’s performance,
G. Sawalha, I. Taj, and A. Shoufan, “Analyzing student prompts and their effect on chatgpt’s performance,” Cogent Education, vol. 11, no. 1, p. 2397200, 2024
2024
-
[21]
Generative artificial intel- ligence and engineering education
A. Johri, A. S. Katz, J. Qadir, and A. Hingle, “Generative artificial intel- ligence and engineering education.” Journal of Engineering Education , vol. 112, no. 3, 2023
2023
-
[22]
Reflections on AI
A. Johri, “Reflections on AI.” [Online]. Available: https://www.ieai.sot.tum.de/wp-content/uploads/2022/05/ Reflections-on-AI-Ethics AdityaJohri-FINAL.pdf
2022
-
[23]
Agentcoder: Multi-agent-based code generation with iterative testing and optimisation,
D. Huang, J. M. Zhang, M. Luck, Q. Bu, Y . Qing, and H. Cui, “Agentcoder: Multi-agent-based code generation with iterative testing and optimisation,” arXiv preprint arXiv:2312.13010 , 2023. 11
2023 arXiv
-
[24]
MetaGPT: Meta programming for multi-agent collaborative framework,
S. Hong, X. Zheng, J. Chen, Y . Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al., “MetaGPT: Meta programming for multi-agent collaborative framework,” arXiv preprint arXiv:2308.00352, 2023
2023 arXiv
-
[25]
Protagents: protein discovery via large language model multi-agent collaborations combining physics and machine learning,
A. Ghafarollahi and M. J. Buehler, “Protagents: protein discovery via large language model multi-agent collaborations combining physics and machine learning,” Digital Discovery, 2024
2024
-
[26]
Co-navGPT: Multi-robot cooperative visual semantic navigation using large language models,
B. Yu, H. Kasaei, and M. Cao, “Co-navGPT: Multi-robot cooperative visual semantic navigation using large language models,” arXiv preprint arXiv:2310.07937, 2023
2023 arXiv
-
[27]
Medagents: Large language models as collaborators for zero-shot medical reasoning,
X. Tang, A. Zou, Z. Zhang, Z. Li, Y . Zhao, X. Zhang, A. Cohan, and M. Gerstein, “Medagents: Large language models as collaborators for zero-shot medical reasoning,” arXiv preprint arXiv:2311.10537 , 2023
2023 arXiv
-
[28]
Camel: Communicative agents for “mind
G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, “Camel: Communicative agents for “mind” exploration of large language model society,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023
2023
-
[2017]
Available: https://proceedings.neurips.cc/paper files/ paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
[Online]. Available: https://proceedings.neurips.cc/paper files/ paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
2017
-
[2024]
Available: https://deepmind.google/technologies/gemini/
[Online]. Available: https://deepmind.google/technologies/gemini/
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.