REVIEW 4 major objections 6 minor 23 references
Are LLMs complicated ethical dilemma analyzers?
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Large language models, prompted into a five-section format, align more closely with expert ethical analyses than non-expert humans do, yet they remain weak at historical grounding and nuanced resolution strategies.
desk verdict Useful benchmark, but the central human-vs-LLM comparison is undercut by LLM preprocessing of both sides; deserves revision, not dismissal. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a fixed five-section response format paired with a composite similarity score. Each model's answer, each expert reference, and each non-expert key-factor statement is rendered in the same structured outline, and quality is quantified as a weighted sum of four metrics: BLEU, Damerau-Levenshtein distance, TF-IDF cosine similarity, and Universal Sentence Encoder semantic similarity. The weights are not arbitrary: candidate metrics were ranked against a hand-made ordering of ten responses, the best metric from each category was kept, and final weights came from an inverted-softmax transform plus analytic hierarchy process pairwise comparisons. The five-section format makes component-wise diagnosis possible; the composite metric turns 'alignment with expert opinion' into a single comparable number.
What would settle it
Collect raw, unedited non-expert responses to the same 196 dilemmas, score them with the same composite metric against the same expert references, and compare the gap to the LLM scores; if the gap shrinks or reverses, the reported human-versus-LLM difference is largely an artifact of LLM preprocessing.
Extended reading notes
Core claim
The central claim is that LLM performance on ethical dilemmas can be measured by structured alignment with expert references, and that under this measure LLMs outperform non-expert humans. The benchmark contains 51 experimental dilemmas with expert opinions, supplemented by 145 more cases, and each expert response is reorganized into a five-section format; four non-expert human responses were collected for the key-factors section only. Using a weighted composite of BLEU, Damerau-Levenshtein distance, TF-IDF cosine similarity, and Universal Sentence Encoder similarity, with weights derived from manual rankings and analytic hierarchy process, the paper finds all four models scoring in the 0.41 to 0.45 range, GPT-4o-mini the most consistent, and Claude-3.5-Sonnet the weakest, particularly on resolution strategies. Non-expert humans score lower on lexical alignment but come closer on semantic similarity, suggesting intuitive but unstructured moral insight.
Load-bearing premise
The load-bearing premise is that the text being compared is genuinely expert and human content: non-expert answers were expanded by an LLM prompt into a 'well-organized key factor' and expert opinions were restructured by LLMs into the five-section format, so the alignment scores largely measure how well LLM-shaped text matches LLM-shaped references rather than how LLM reasoning compares with human reasoning.
Editorial extensions
If this is right
- Structured, prompt-driven LLM answers align with expert references more than non-expert human responses do, at least in the key-factors section where both are directly compared.
- The five-section benchmark can serve as a training signal, making it straightforward to test whether fine-tuning improves the weakest sections, especially resolution strategies and historical perspectives.
- The reported model ordering is benchmark-specific: GPT-4o-mini's consistency across sections does not by itself generalize to other tasks or evaluation metrics.
- Current LLMs are not yet reliable proxies for expert ethical reasoning whenever historical grounding and nuanced resolution strategies matter.
Reading between the lines
- Because the 'human' baseline was itself LLM-expanded, the real gap between raw human prose and LLM output is probably larger on lexical metrics and smaller on structural ones than the paper reports; scoring unedited human text would settle this.
- The metric-selection step uses manual rankings of outputs from a single model, so the chosen weights may not be stable across models; rerunning the inversion analysis with rankings from several models and human judges would test whether the model ordering survives.
- Treating expert summaries as the reference defines alignment as correctness; on dilemmas where experts disagree, a multi-reference or judged-debate evaluation would separate conformity from genuine moral quality.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a benchmark dataset of 196 real-world research-ethics dilemmas with expert opinions, decomposes both expert and LLM responses into five fixed sections (Introduction, Key Factors, Historical & Theoretical Perspectives, Resolution Strategies, Key Takeaways), and evaluates four LLMs (GPT-4o-mini, Claude-3.5-Sonnet, Deepseek-V3, Gemini-1.5-Flash) against LLM-processed expert references using a weighted composite of BLEU, Damerau-Levenshtein distance, TF-IDF cosine similarity, and Universal Sentence Encoder similarity. Metric weights are derived from manual rankings via inversion counts and an AHP judgment matrix. Non-expert human responses, collected from four participants, are also compared on the Key Factors section. The headline claims are that LLMs generally outperform non-expert humans in lexical and structural alignment, that GPT-4o-mini is the most consistent model, and that all models struggle with historical grounding and nuanced resolution strategies. The appendices contain the exact prompts used to preprocess expert opinions, extend human answers, and generate LLM responses.
Significance. The dataset and code are public, and the paper addresses a genuinely important question: whether LLMs can serve as proxies for human ethical reasoning. The structured five-section decomposition is a reasonable way to make open-ended moral reasoning comparable, and the use of multiple complementary metrics is a sensible starting point. However, the central measurement is confounded by LLM preprocessing on both sides of the comparison: expert references are restructured by an LLM (Appendix A.1), and human answers are extended by an LLM into 'well-organized key factors' (Appendix A.2). The example outputs in A.1 and A.3 are nearly identical, so lexical and structural overlap between LLM outputs and LLM-processed references may reflect shared prompt conventions rather than ethical reasoning quality. In addition, the metric weights are fit to the authors' own manual rankings without held-out validation, and no significance testing is reported for the small between-model differences.
major comments (4)
- [§5.2, Appendices A.1–A.2] The headline comparison in §5.2 is not LLM-vs-human reasoning. Appendix A.2's system prompt instructs the model to 'extend a short opinion into a well-organized key factor', and the user prompt caps the output at three times the human answer; Appendix A.1 asks an LLM to 'structure the expert's perspective' into the same five-section framework used for LLM answers. The 'human' responses evaluated in Figure 4 are therefore LLM-expanded paragraphs, and the 'expert' references are LLM restructurings. The claim that human responses 'are less structured' is directly contradicted by this preprocessing, which explicitly organizes human opinions into well-formed paragraphs. The observed lexical and structural gaps may reflect shared prompt-formatting conventions between LLM-generated answers and LLM-processed references rather than differences in ethical reasoning. The manuscript should either re-run the comparison on raw human text and raw expert text, or explicitly reframe all conclusions as comparisons among LLM-formatted texts.
- [§4.2] The metric selection and weight-calculation procedure is fit entirely to the authors' own manual rankings. Metrics are selected by inversion counts on 10 responses from Gemini, and weights are derived from 20 responses from Claude, with no held-out validation, no inter-annotator agreement, and no sensitivity analysis. The AHP judgment matrix entries are given without justification, and no consistency ratio is reported. The composite score in Table 1 may therefore be overfit to idiosyncratic manual rankings, and it is unclear whether the reported model ordering would survive alternative weights. The authors should validate the metric weights on a held-out set and report the sensitivity of the model ranking to reasonable variations in the weights.
- [§5.1, Table 1] Table 1 reports average scores that differ by only 0.01–0.04 across models (e.g., GPT-4o-mini 0.4525 vs. Gemini 0.4460), yet the text claims that Sonnet 'significantly struggles' and that GPT-4o-mini 'surpasses all other models'. No standard errors, confidence intervals, or significance tests are provided. Given the small differences and the confounded reference standard, these comparative claims are not supported. The authors should report per-dilemma variance and paired or bootstrap significance tests, or soften the comparative conclusions accordingly.
- [§5.1] Section 5.1 describes scores in the range 0.40–0.60 as 'accuracy'. The composite measure is a weighted similarity to LLM-restructured expert references, not a classification accuracy, and no chance-level or random-baseline comparison is provided. The text should replace 'accuracy' with 'similarity score' and give a baseline (e.g., random sentence permutations or a trivial template response) to make the magnitudes interpretable.
minor comments (6)
- [Title, Abstract, §1] The title and abstract contain grammatical and typographical issues ('LLMs' instead of 'LLM's', 'Are LLMs complicated ethical dilemma analyzers?'), and §3.1 has '51 induplicate cases' which should be '51 unique cases' or '51 duplicate-free cases'.
- [§3.2, Abstract] The abstract says 'four non-expert human participants' were collected; §3.2 says 'four non-expert individuals per dilemma'. Please clarify whether the same four participants answered all 51 experimental dilemmas or different participants were used, and report their demographic or selection information if available.
- [Figure 4] The caption says 'processed1/2/3/4 are four different non-expert individual human data providers', but the figure itself does not label which panel is which, and it is unclear whether the right panel is the Key-Factors-only LLM score or the full five-section score. Please make the panels and the meaning of 'processed' explicit.
- [§4.2, Eq. (1)] The weight formula after Eq. (1) uses s_i but does not define n or state explicitly that min/max are taken over the four categories; the phrase 'inverted softmax' is non-standard and should be clarified. The mapping of rows/columns of the AHP matrix to semantic, n-gram, cosine, and lexical metrics is also not given.
- [§4.2] Damerau-Levenshtein is a distance, not a similarity, yet it appears as one of the four metrics in a weighted sum where higher values are described as better. Please specify the normalization and direction used so that the composite score is unambiguous.
- [References] Several in-text source references are raw URLs or missing citation keys (e.g., 'Georgia Clinical & Translational Science Alliance' and 'Online Ethics Center' in §1), and the reference list does not consistently use a single format. Please standardize the citations.
Circularity Check
The human and expert baselines are both LLM-processed, so the central LLM-vs-human comparison partly measures LLM-to-LLM text similarity rather than human reasoning.
-
self definitional
[Appendix A.2 (Human Evaluation Data Preprocessing); applied in Section 5.2]
"Your task is to extend a short opinion into a well-organized key factor for ethical dilemmas. ... Make sure the output is in utf-8 format. Make sure the size of the output does not exceed three times the human answer."
The texts labeled 'non-expert human responses' and scored in Figure 4 are not raw human answers; each was rewritten and expanded by an LLM before evaluation. The paper then concludes that 'human responses ... exhibit greater lexical variability and less alignment with expert references' (Section 5.2), but this is a property of LLM-expanded text, not of spontaneous human writing. The preprocessing explicitly imposes well-organized structure on the human answers, so the claimed human-LLM structural gap is partly an artifact of the pipeline rather than an independently measured difference in ethical reasoning.
-
self definitional
[Section 4.1 (Prompting for the Answers of LLMs); Appendix A.1 (Expert Data Preprocessing)]
"each LLM was instructed to produce outputs adhering to the aforementioned five-section format: Introduction, Key Factors in Consideration, Historical and Theoretical Perspectives, Proposed Resolution Strategies, and Key Takeaways. Subsequently, during the evaluation phase, the responses generated by each LLM were quantitatively assessed against four distinct sets of expert-opinion summaries, each preprocessed independently by different LLMs."
The 'expert references' against which LLMs are scored are themselves LLM restructurings: Section 4.1 says the responses were assessed against expert-opinion summaries 'preprocessed independently by different LLMs,' and Appendix A.1's prompt asks the LLM to 'structure the expert's perspective into a detailed analytical framework' using the same five-section format and nearly the same example text as Appendix A.3's LLM generation prompt. High 'alignment with expert references' therefore partly measures agreement with an LLM-generated template, not with the original expert text.
1 more flagged steps
-
fitted input called prediction
[Section 4.2 (Metrics Setup)]
"To do so, we generated ten responses from the Gemini model (selected arbitrarily) and manually ranked these responses from best to worst, allowing for tied ranks where appropriate. ... we manually ranking 20 outputs from Claude (selected arbitrarily) and tested each category's generated ordering to calcuate inversions."
The composite metric is not a parameter-free evaluation instrument: its component selection and final weights are optimized against the authors' own manual rankings of outputs from two of the same models later scored. Presenting the resulting weighted scores as objective 'alignment with expert references' treats a hand-fitted rubric as an independent measure. This step is ancillary to the human-baseline issue, but it reinforces that the reported rankings are tuned rather than derived from a neutral gold standard.
full rationale
The central comparison in Section 5.2 is not self-contained because both sides of the comparison pass through LLMs before scoring. Non-expert human answers are expanded by an LLM into 'well-organized key factor[s]' (Appendix A.2), and the expert references are LLM restructurings produced with the same five-section template used to prompt the evaluated models (Appendix A.1; Section 4.1 states these summaries were 'preprocessed independently by different LLMs'). The headline result that LLMs outperform non-expert humans in lexical and structural alignment is therefore partly a comparison of LLM-generated text against LLM-processed text, so the measured gaps may reflect shared prompt-formatting conventions rather than differences in ethical reasoning. In addition, the composite metric weights are fitted to the authors' manual rankings of a small number of outputs from two of the evaluated models (Section 4.2), so the scoring rubric is not an independent gold standard. These issues make the main quantitative claim partially self-referential: the evaluation pipeline shapes both the 'human' baseline and the 'expert' reference with the same kind of system under test. The conclusion is not forced by definitional identity alone, and the component-wise observations about historical grounding and resolution strategies retain some independent content, so the appropriate score is 6 (partial circularity), not higher.
Assumptions & free parameters
free parameters (3)
- Metric weights (semantic, N-gram, cosine, lexical) =
0.5386, 0.1547, 0.2299, 0.0768
- AHP judgment matrix entries =
matrix shown in Section 4.2
- Section weights (five subparts) =
equal (implicit)
assumptions (4)
- domain assumption LLM-based preprocessing preserves expert opinion content and structure faithfully.
- ad hoc to paper Manual rankings of 10-20 LLM responses are a valid ground truth for metric selection.
- domain assumption The four selected metrics jointly capture the relevant dimensions of ethical reasoning quality.
- ad hoc to paper Non-expert human responses, once LLM-extended, remain representative of human judgment.
Cite this review
Pith. "Pith review of Are LLMs complicated ethical dilemma analyzers?." pith.science (2026). https://pith.science/paper/7N6T32U7
@misc{pith2026250508106,
author = {Pith},
title = {Pith review of: Are LLMs complicated ethical dilemma analyzers?},
year = {2026},
howpublished = {\url{https://pith.science/paper/7N6T32U7}},
note = {Machine review of arXiv:2505.08106}
}
read the original abstract
One open question in the study of Large Language Models (LLMs) is whether they can emulate human ethical reasoning and act as believable proxies for human judgment. To investigate this, we introduce a benchmark dataset comprising 196 real-world ethical dilemmas and expert opinions, each segmented into five structured components: Introduction, Key Factors, Historical Theoretical Perspectives, Resolution Strategies, and Key Takeaways. We also collect non-expert human responses for comparison, limited to the Key Factors section due to their brevity. We evaluate multiple frontier LLMs (GPT-4o-mini, Claude-3.5-Sonnet, Deepseek-V3, Gemini-1.5-Flash) using a composite metric framework based on BLEU, Damerau-Levenshtein distance, TF-IDF cosine similarity, and Universal Sentence Encoder similarity. Metric weights are computed through an inversion-based ranking alignment and pairwise AHP analysis, enabling fine-grained comparison of model outputs to expert responses. Our results show that LLMs generally outperform non-expert humans in lexical and structural alignment, with GPT-4o-mini performing most consistently across all sections. However, all models struggle with historical grounding and proposing nuanced resolution strategies, which require contextual abstraction. Human responses, while less structured, occasionally achieve comparable semantic similarity, suggesting intuitive moral reasoning. These findings highlight both the strengths and current limitations of LLMs in ethical decision-making.
Figures
Reference graph
Works this paper leans on
-
[1]
spacy: Linguistic features - similarity
Explosion AI. spacy: Linguistic features - similarity. https://spacy.io/usage/linguistic-features#similarity, 2024. Accessed: 2025-05-11
work page 2024
-
[2]
Anthropic. Claude 3.5 sonnet. https://www.anthropic.com/news/claude-3-5-sonnet, 2024. Accessed: 2025-05-11
work page 2024
-
[3]
Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. Universal sentence encoder. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.\ 169--174. Associ...
-
[4]
Wikipedia Contributors. Overlap coefficient. https://en.wikipedia.org/wiki/Overlap_coefficient, 2024. Accessed: 2025-05-11
work page 2024
-
[5]
Fred J. Damerau. A technique for computer detection and correction of spelling errors. Communications of the ACM, 7 0 (3): 0 171--176, 1964. doi:10.1145/363958.363994
arXiv 1964
-
[6]
Google DeepMind. Gemini 1.5 flash. https://deepmind.google/technologies/gemini/, 2024. Accessed: 2025-05-11
work page 2024
-
[7]
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huaj...
arXiv 2024
-
[8]
Ethical dilemmasin scientific research and professional integrity
Georgia Clinical & Translational Science Alliance . Ethical dilemmasin scientific research and professional integrity. https://georgiactsa.org/research/regulatory-knowledge-support/ethics-center.html. Accessed: 2025-05-11
work page 2025
Show all 23 references
-
[9]
Étude comparative de la distribution florale dans une portion des alpes et des jura
Paul Jaccard. Étude comparative de la distribution florale dans une portion des alpes et des jura. Bulletin de la Société Vaudoise des Sciences Naturelles, 37: 0 547--579, 1901
1901
-
[10]
Triage: Ethical benchmarking of ai models through mass casualty simulations, 2024
Nathalie Maria Kirch, Konstantin Hebenstreit, and Matthias Samwald. Triage: Ethical benchmarking of ai models through mass casualty simulations, 2024. URL https://arxiv.org/abs/2410.18991
2024 arXiv
-
[11]
Kusner, Yu Sun, Nicholas I
Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, and Kilian Q. Weinberger. From word embeddings to document distances. In Proceedings of the 32nd International Conference on Machine Learning, pp.\ 957--966. PMLR, 2015
2015
-
[12]
Fairmindsim: Alignment of behavior, emotion, and belief in humans and llm agents amid ethical dilemmas, 2024
Yu Lei, Hao Liu, Chengxing Xie, Songjia Liu, Zhiyu Yin, Canyu Chen, Guohao Li, Philip Torr, and Zhen Wu. Fairmindsim: Alignment of behavior, emotion, and belief in humans and llm agents amid ethical dilemmas, 2024. URL https://arxiv.org/abs/2410.10398
2024 arXiv
-
[13]
Rouge: A package for automatic evaluation of summaries
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp.\ 74--81, Barcelona, Spain, 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013/
2004
-
[14]
Collections
Online Ethics Center . Collections. https://onlineethics.org/collections. Accessed: 2025-05-11
2025
-
[15]
Gpt-4o-mini model
OpenAI. Gpt-4o-mini model. https://platform.openai.com/docs/models/gpt-4o, 2024. Accessed: 2025-05-11
2024
-
[16]
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp.\ 311--318. Association for Computational Linguistics, 20...
2002
-
[17]
Thomas L. Saaty. The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. McGraw-Hill, New York, 1980. ISBN 978-0070543713
1980
-
[18]
Term-weighting approaches in automatic text retrieval, volume 24
Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval, volume 24. Elsevier, 1988. doi:10.1016/0306-4573(88)90021-0
1988 doi
-
[19]
Soft similarity and soft cosine measure: Similarity of features in vector space model
Grigori Sidorov, Alexander Gelbukh, David Pinto, Pavel Rosso, and Francisco Velasquez. Soft similarity and soft cosine measure: Similarity of features in vector space model. In Computing Research Repository, volume abs/1412.6035, 2014. URL https://arxiv.org/abs/1412.6035
2014 arXiv
-
[20]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
-
[21]
@esa (Ref
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
-
[22]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
-
[23]
I verified the compound this morning, so we’re on our way
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...
1980
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.