{"id":"1b2e82d4-55b0-4806-90be-207702f79467","arxiv_id":"2507.17216","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"LLMs align with human moral judgments only under high consensus, concentrate on a narrow set of moral values, and the profile-based prompting method's reported improvement is evaluated in-sample.","lead":"This paper compares how large language models and humans judge real-world moral dilemmas, using a new dataset of 1,618 cases. It reports that models align with humans mainly on uncontroversial dilemmas and proposes a prompting method that narrows the gap by 64.3%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DMP's 64.3% alignment gain is measured in-sample: the value profiles G_t are fit to all human rationales, including the exact dilemmas later used to score DMP, so the headline improvement likely reflects test-set leakage rather than a generalizable steering effect.","rationale":"The reader's weakest assumption identifies exactly the load-bearing issue: the central claim about DMP's 64.3% improvement depends on the value profiles being a valid conditioning signal, but the paper estimates them from the same human rationales used for evaluation. I agree with that assessment. The paper's own text confirms the leakage: G0 is 'aggregated across all human rationales in the dataset' and each dilemma's profiles are sampled from G_{t_i}, with evaluation following Section 3.1 on the same dilemmas. This does not make the descriptive findings or the dataset less useful, and a held-out evaluation could still support a weaker version of the claim, so the appropriate stance remains conditional rather than outright rejection. The concrete test above would settle the matter. No additional objections beyond the reader's are needed; the lack of alpha sensitivity analysis and error bars are secondary but should be included in a revision.","tokens_in":14446,"tokens_out":4157,"duration_ms":43570,"concrete_test":"Estimate G0 and all topic-specific G_t using only training dilemmas (e.g., topic-stratified 5-fold cross-validation or leave-one-dilemma-out), then evaluate DMP on held-out dilemmas with the same N_i and prompt setup, comparing against the model council. If the held-out absolute difference remains close to 22pp rather than dropping to roughly 8pp, the 64.3% improvement is an artifact of in-sample profiling. Report results across alpha in {1, 5, 10, 50} and bootstrap confidence intervals to check sensitivity and variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 defines G0 as the empirical frequency over all human rationales in the dataset and fits G_t ~ Dirichlet(alpha*G0) per topic. It then samples N_i profiles for each dilemma d_i from G_{t_i} and evaluates DMP with the same procedure as Section 3.1, which computes Delta_i between human and LLM judgment distributions on those same dilemmas. This means the conditioning profiles for dilemma i are built from human rationales that include dilemma i's own rationales. The model council baseline receives no such information, so the reported 22pp-to-8pp reduction (64.3%) confounds genuine steering with leakage of the human answer distribution into the prompt. No held-out split, cross-validation, or leave-one-out analysis is reported, and the Limitations section does not mention this. The free parameter alpha=10 also lacks sensitivity analysis. RQ1/RQ2 descriptive findings are not affected because they do not use G_t; the load-bearing problem is specific to the central RQ3 claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Moral Dilemma Dataset (MDD), 1,618 real-world moral dilemmas from r/AmITheAsshole with human binary judgments and free-text rationales, and studies distributional alignment between LLM and human moral judgments. RQ1 shows that LLM-human alignment degrades as human consensus decreases. RQ2 builds a 60-value taxonomy from human and LLM rationales using the Value Kaleidoscope and reports that LLMs concentrate their value mentions on a narrow top-10 set (81.6% vs. 35.2% for humans). RQ3 proposes Dynamic Moral Profiling (DMP), a Dirichlet-based method that conditions LLM prompts on value profiles sampled from topic-specific distributions derived from all human rationales; the paper reports a 64.3% reduction in the average absolute judgment difference relative to the model council baseline and an increase in value diversity. The central RQ3 claim is, however, compromised by a leakage problem in the evaluation: the value profiles are estimated from all human rationales, including the same dilemmas used to measure alignment, so the reported improvement conflates genuine steering with access to the human answer distribution.","tokens_in":14654,"tokens_out":5046,"duration_ms":51489,"significance":"If the leakage issue were resolved, this paper would be a solid empirical contribution. The MDD dataset fills a gap by providing ecologically valid, distribution-level moral judgments and rationales, and the RQ1 finding that alignment drops sharply with lower human consensus is a clean, well-supported observation. The RQ2 value-diversity result is plausible and useful, though it depends on the LLM-based Value Kaleidoscope extractor and the clustering pipeline. The DMP method is a creative attempt to steer pluralistic model outputs, and the theory-driven MFT baselines are a reasonable control. However, the headline 64.3% improvement is not interpretable as a generalizable steering effect because the conditioning profiles are built from the test dilemmas themselves. The paper would be strengthened by a held-out or leave-one-dilemma-out evaluation, a sensitivity analysis for the free parameters, and a more precise specification of the generative procedure.","major_comments":[{"comment":"The evaluation of DMP is affected by test-set leakage. In Section 5, the base measure G0 is defined as the empirical frequency distribution over all human rationales in the dataset, and the topic-specific Gt is fit via Dirichlet(alpha*G0) using those same rationales. Profiles for dilemma di are then sampled from Gti and used to condition the model, and alignment is measured on the same dilemma di with the procedure of Section 3.1. Thus the conditioning profile for each dilemma includes value-frequency information drawn from that dilemma's own human rationales. The model council baseline receives no such information, so the reported reduction from 22pp to 8pp (64.3% improvement) confounds the steering effect with leakage of the human judgment distribution. RQ1 and RQ2 are not affected because they do not use Gt, but the central RQ3 claim is not supported by the reported experiment. The authors should evaluate DMP on held-out dilemmas or use leave-one-dilemma-out cross-validation when estimating G0 and Gt, and report the resulting alignment and diversity metrics.","section":"Section 5 (DMP evaluation) and Section 5.1"},{"comment":"The concentration parameter alpha is set to 10 with no sensitivity analysis. Because DMP's behavior is directly controlled by how strongly the topic-specific profiles are allowed to deviate from the global prior, the reported 64.3% improvement and the diversity gains could be highly sensitive to this arbitrary choice. The authors should report the main results across a range of alpha values (e.g., 1, 5, 10, 20, 50) and show that the qualitative conclusions are stable.","section":"Section 5 (Dirichlet concentration parameter)"},{"comment":"The generative definition of Gt is underspecified. The text states 'we fit a topic-specific distribution Gt ∼ Dirichlet(alpha*G0) for each topic t', but it does not state whether Gt is a posterior estimate conditioned on the topic's rationales, a single random draw, or a MAP estimate, nor does it explain how topics are defined and how many topics exist. This ambiguity prevents exact reproduction of the DMP procedure and should be resolved with a precise algorithm or pseudocode.","section":"Section 5 (generative specification)"}],"minor_comments":[{"comment":"The text 'reported results for temperature 1 due to its superior performance (see Appendix 4 for comparison)' should refer to the actual appendix table, namely Table 4 in Appendix A.2.1.","section":"Section 3.2"},{"comment":"The phrase 'Evaluation fall into five standard categories' should be 'Evaluations fall into five standard categories'.","section":"Section 2.1"},{"comment":"The phrase 'The buckets where almost balanced' should be 'The buckets were almost balanced'.","section":"Section 2.1"},{"comment":"The phrase 'thee top 10 values account for 81.6%' contains a typo; it should be 'the top 10 values'.","section":"Section 4.2"},{"comment":"The phrase 'low-consensus dilemmans' should be 'low-consensus dilemmas'.","section":"Section 5.1"},{"comment":"The phrase 'All other models exhibited notably worst alignment' should be 'notably worse alignment'.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The dataset and RQ1/RQ2 results are solid contributions, and the DMP idea is interesting. The central problem is the leakage in the DMP evaluation, which I believe is fixable with a proper held-out evaluation. If the authors can show that the improvement holds under a leave-one-dilemma-out or held-out split and provide the requested sensitivity analysis, I would be willing to support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The dataset and the RQ1/RQ2 findings are the real value here. The Moral Dilemma Dataset is a genuinely useful resource: real-world dilemmas with full distributions of human judgments, not just majority labels. The finding that alignment degrades as consensus drops is robust and worth having. The value-taxonomy analysis is more dependent on the LLM-based extractor and clustering, but the top-10 concentration gap (81.6% vs 35.2%) is a clear and plausible pattern. Those parts of the paper deserve serious attention.\n\nThe problem is RQ3. The value profiles G0 and G_t are estimated from all human rationales in the dataset, including the rationales for the exact dilemmas used to evaluate DMP. So when DMP conditions on those profiles, it is effectively given information about the human answer distribution it is then scored against. The model council baseline gets no such information, so the reported 22pp-to-8pp reduction (64.3%) confounds genuine steering with leakage. There is no held-out split, no leave-one-out analysis, and the Limitations section does not mention this. The free parameter alpha=10 also lacks any sensitivity analysis.\n\nThis is not a minor quibble. It means the headline claim about DMP is not supported as stated. But it is also fixable. A proper held-out or cross-validated evaluation could still show a real effect, and the method is interesting enough to warrant that check. The descriptive RQ1 and RQ2 results are not affected by the circularity.\n\nI would send this to peer review rather than desk reject. The dataset and descriptive findings are solid, and the DMP idea deserves a clean test. Reviewers should push for a proper evaluation setup, error bars, sensitivity analysis for alpha, and a data/code release. If the authors fix the evaluation, the paper could be competitive. As is, the abstract overstates what is actually shown.","headline":"Solid dataset and descriptive findings; the DMP improvement claim is likely inflated by test-set leakage and needs a held-out evaluation before it can be trusted.","tokens_in":15162,"tokens_out":1461,"would_cite":true,"duration_ms":16838,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LLMs match human moral judgments only under high consensus, and that a prompting method called Dynamic Moral Profiling cuts the judgment-distribution gap by 64.3% while also broadening the moral values models express.","keywords":["moral judgment","distributional alignment","value pluralism","large language models","moral dilemmas","Dirichlet sampling","prompting","moral values"],"falsifier":"Estimate the global and topic value profiles on a training split of the Moral Dilemma Dataset and apply DMP only to held-out dilemmas; if the average absolute difference no longer falls from 22 to 8 percentage points, the reported improvement is at least partly an artifact of conditioning on the evaluation data.","tokens_in":14232,"feed_emoji":"⚖️","tokens_out":7284,"duration_ms":63823,"temperature":0.7,"pith_summary":"People increasingly ask large language models for moral advice, but models are usually judged against majority opinions rather than the full spread of human views. This paper builds a dataset of 1,618 real-world dilemmas paired with distributions of human judgments and free-text rationales, and measures how closely LLMs reproduce the entire distribution, not just the majority label. It finds that models align with humans only when human consensus is high, and that in disputed dilemmas models default toward a single judgment while relying on a narrow set of moral values. The paper argues that a prompting method called Dynamic Moral Profiling, which samples value profiles from a Dirichlet distribution fit to human rationales, closes much of this gap: average disagreement falls by 64.3%, with the largest gains in low-consensus cases. If this holds, value-conditioned prompting is a cheap way to make LLM moral advice more pluralistic.","feed_headline":"Prompting with human value profiles cuts LLM-human moral gap by 64%","feed_subtitle":"A Dirichlet-sampled moral value profile steers models to match human judgment splits and widen value diversity.","key_machinery":"The load-bearing object is the dynamic moral profile: a topic-specific probability distribution $G_t$ over a 60-value taxonomy, constructed by taking the global empirical value distribution $G_0$ over all human rationales and fitting $G_t \\sim \\mathrm{Dirichlet}(\\alpha G_0)$ per topic with $\\alpha = 10$. For each dilemma, $N_i$ profiles are sampled from the multinomial $G_{t_i}$, each profile is a short list of top values with normalized importance weights, and the profile is inserted into the prompt so the model judges the dilemma through that value lens. The mechanism works on two levels: it injects human value diversity into individual generations, and because the sampled profiles vary across the $N_i$ queries, it reshapes the aggregated model judgment distribution toward the human one.","core_discovery":"The central discovery is that the divergence between LLM and human moral judgment is distributional rather than only a matter of matching the majority answer. Across the 1,618 dilemmas, the average absolute difference between human and LLM proportions of Acceptable judgments is 22 percentage points for the best baseline, a council of models, and it rises to roughly 30 points in the most disputed dilemmas. Using a 60-value taxonomy built from 3,783 value expressions, the paper shows that LLMs place 81.6% of their value mentions in their top 10 values, versus 35.2% for humans, indicating a narrower moral vocabulary. Dynamic Moral Profiling conditions each model response on a profile of values sampled from a topic-level Dirichlet distribution over the taxonomy; this reduces the average absolute difference from 22 to 8 percentage points, a 64.3% improvement, and raises normalized value entropy from 0.46 to 0.52, closer to the human 0.57.","pith_inferences":["The paper estimates the value profiles on the same dilemmas used to measure alignment; a held-out split would test whether the 64.3% improvement survives when profiles are built from different dilemmas, and the paper reports no such check.","The taxonomy is derived from one online community, so the pluralism DMP injects is pluralism within that community's value space; applying the procedure to other cultural contexts would likely require rebuilding the taxonomy rather than reusing the 60 values.","Because each query gets one sampled profile, an extension would be to compress the plural distribution into a single response that reports the spread of moral perspectives instead of one verdict.","A further testable extension is to measure downstream effects, such as whether people exposed to DMP-generated rationales make more considered or more diverse decisions than people exposed to standard model outputs."],"forward_implications":["With the reported 64.3% improvement, a prompt-only intervention reduces LLM-human moral judgment disagreement without fine-tuning or preference data.","The biggest gains occur in low-consensus dilemmas, precisely the cases where people are most likely to seek moral advice, because DMP breaks models' default to a single Acceptable judgment.","DMP broadens the value distribution in model rationales: the top-10 value concentration drops from 81.6% to 51.3%, and value entropy approaches the human baseline.","The failure of Moral Foundations Theory baselines suggests that the specificity of the data-driven 60-value taxonomy, not just any value framing, is what carries the alignment improvement.","Since DMP samples a fresh value profile for each query, its pluralistic alignment depends on repeated sampling rather than a single user-facing response."],"supporting_citations":[{"why":"Supplies the Value Kaleidoscope system used to extract the moral value expressions that are clustered into the 60-value taxonomy.","marker":"Sorensen et al., 2024a"},{"why":"Provides the Dirichlet-multinomial formulation that Dynamic Moral Profiling adapts to sample topic-level value profiles.","marker":"Blei et al., 2003"},{"why":"Defines Moral Foundations Theory, whose six foundations are used as the theory-driven baseline that DMP is compared against.","marker":"Graham et al., 2013"},{"why":"Supports the panel-of-models evaluation strategy used to build the model council baseline.","marker":"Verga et al., 2024"},{"why":"Introduces the council-of-models idea for benchmarking subjective tasks, the strongest baseline DMP improves on.","marker":"Zhao et al., 2024a"},{"why":"Provides the extended Moral Foundations Dictionary used to compute the MFT-based value profiles for comparison.","marker":"Hopp et al., 2021"},{"why":"Frames distributional alignment of LLM outputs, the evaluation lens the paper applies to moral judgments.","marker":"Meister et al."}],"fun_headline_variants":["LLMs mirror human morals only when humans agree","Moral alignment gap widens when humans disagree","64% better moral fit: LLMs tuned to human value profiles","Pluralistic moral gap: LLMs narrow on values, humans diverse","Dynamic moral profiling shrinks LLM-human judgment gap by 64%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the value profiles used to steer the model are estimated from the same dilemmas that are then used to measure alignment, so the reported 64.3% improvement assumes no leakage from the test dilemmas into the profiles; a held-out evaluation could shrink the gain.","fun_headline_variants_meta":{"raw":{"variants":["LLMs mirror human morals only when humans agree","Moral alignment gap widens when humans disagree","64% better moral fit: LLMs tuned to human value profiles","Pluralistic moral gap: LLMs narrow on values, humans diverse","Dynamic moral profiling shrinks LLM-human judgment gap by 64%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000606,"raw_usage":{"total_tokens":2837,"prompt_tokens":971,"completion_tokens":1866,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":1795}},"tokens_in":587,"tokens_out":1866,"duration_ms":14464,"temperature":1.0,"reasoning_tokens":1795,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:53:42.278794+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Estimate the global and topic value profiles on a training split of the Moral Dilemma Dataset and apply DMP only to held-out dilemmas; if the average absolute difference no longer falls from 22 to 8 percentage points, the reported improvement is at least partly an artifact of conditioning on the evaluation data.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the extended Moral Foundations Dictionary used to compute the MFT-based value profiles for comparison."}],"review_version":1}