REVIEW 5 cited by
The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As large language models (LLMs) become increasingly integrated into society, their alignment with human morals is crucial. To better understand this alignment, we created a large corpus of human- and LLM-generated responses to various moral scenarios. We found a misalignment between human and LLM moral assessments; although both LLMs and humans tended to reject morally complex utilitarian dilemmas, LLMs were more sensitive to personal framing. We then conducted a quantitative user study involving 230 participants (N=230), who evaluated these responses by determining whether they were AI-generated and assessed their agreement with the responses. Human evaluators preferred LLMs' assessments in moral scenarios, though a systematic anti-AI bias was observed: participants were less likely to agree with judgments they believed to be machine-generated. Statistical and NLP-based analyses revealed subtle linguistic differences in responses, influencing detection and agreement. Overall, our findings highlight the complexities of human-AI perception in morally charged decision-making.
Forward citations
Cited by 5 Pith papers
-
The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions
LLMs attribute moral responsibility like humans but refuse to act on it in scarce-resource allocation, defaulting to random choice instead of favoring the less-culpable patient.
-
Localizing Persona Representations in LLMs
Persona information is most separable in the final third of LLM layers, and in Llama3's last layer ethical personas share 17.6% of salient activations while political personas have 2.1% to 5.5% unique activations.
-
Large Language Models Do Not Simulate Human Psychology
LLMs fail to mirror human moral judgments when scenarios are reworded to change meaning, even the human-fine-tuned CENTAUR model.
-
Normative Evaluation of Large Language Models with Everyday Moral Dilemmas
Seven LLMs give different moral verdicts on AITA dilemmas, differ from Redditors, and only in an ensemble approximate human consensus.
-
A Comparative Analysis of Ethical and Safety Gaps in LLMs using Relative Danger Coefficient
The Relative Danger Coefficient is a weighted refusal-based metric for LLM safety, introduced and demonstrated on GPT, Gemini, and DeepSeek models.
Discussion (0). Continue with ORCID to comment.